[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:50 dist-test-slave-49h3 krb5kdc[31991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:50 dist-test-slave-49h3 krb5kdc[31991](info): set up 2 sockets
Aug 08 13:09:50 dist-test-slave-49h3 krb5kdc[31991](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 13:09:50.993800 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:09:51 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:09:51.181298 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42847
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:51.514442 32004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:51.514804 32004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:51.514947 32004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:51.522657 32004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:51.522830 32004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:51.522920 32004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:51.523020 32004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:51.523087 32004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:51.533797 32004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:51.535282 32004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:51.536731 32004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:51.549263 32009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:51.549196 32012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:51.549232 32010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:51.549955 32004 server_base.cc:1034] running on GCE node
I20260808 13:09:51.550765 32004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:51.552289 32004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:51.553503 32004 hybrid_clock.cc:648] HybridClock initialized: now 1786194591553400 us; error 79 us; skew 500 ppm
I20260808 13:09:51.553841 32004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:51 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194591, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:51.556991 32004 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:09:51.558584 32004 webserver.cc:460] Webserver started at http://127.31.59.190:46033/ using document root <none> and password file <none>
I20260808 13:09:51.559310 32004 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:51.559429 32004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:51.559720 32004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:51.563066 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "069a1ba2b8f34447abfd770459a342a6"
format_stamp: "Formatted at 2026-08-08 13:09:51 on dist-test-slave-49h3"
server_key: "815dd420b520a650d2ce8d2f75f94a8d"
server_key_iv: "0efb06b888a656b6f9a4fe845593813f"
server_key_version: "encryptionkey@0"
I20260808 13:09:51.563851 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "069a1ba2b8f34447abfd770459a342a6"
format_stamp: "Formatted at 2026-08-08 13:09:51 on dist-test-slave-49h3"
server_key: "815dd420b520a650d2ce8d2f75f94a8d"
server_key_iv: "0efb06b888a656b6f9a4fe845593813f"
server_key_version: "encryptionkey@0"
I20260808 13:09:51.569293 32004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:09:51.572937 32022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:51.574290 32004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:51.574461 32004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "069a1ba2b8f34447abfd770459a342a6"
format_stamp: "Formatted at 2026-08-08 13:09:51 on dist-test-slave-49h3"
server_key: "815dd420b520a650d2ce8d2f75f94a8d"
server_key_iv: "0efb06b888a656b6f9a4fe845593813f"
server_key_version: "encryptionkey@0"
I20260808 13:09:51.574659 32004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:51.609714 32004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:51.613832 32004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:51.614125 32004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:51.632931 32004 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:42847
I20260808 13:09:51.632947 32077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:42847 every 8 connection(s)
I20260808 13:09:51.634352 32004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:51.637708 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32004
I20260808 13:09:51.638057 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:09:51.638733 32078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:51.640632 31982 external_mini_cluster.cc:1428] Setting key ab77fe0a9f0a8c7af8e4a7055fd360a7
I20260808 13:09:51.651098 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6: Bootstrap starting.
I20260808 13:09:51.654184 32078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:51.655411 32078 log.cc:824] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:51.658762 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6: No bootstrap required, opened a new log
I20260808 13:09:51.668022 32078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } }
I20260808 13:09:51.668496 32078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:51.668607 32078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069a1ba2b8f34447abfd770459a342a6, State: Initialized, Role: FOLLOWER
I20260808 13:09:51.669332 32078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } }
I20260808 13:09:51.669658 32078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:51.669816 32078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:51.670015 32078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 13:09:51 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:51.672962 32078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } }
I20260808 13:09:51.673722 32078 leader_election.cc:302] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 069a1ba2b8f34447abfd770459a342a6; no voters:
I20260808 13:09:51.674432 32078 leader_election.cc:288] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:51.674898 32083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:51.675732 32083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [term 1 LEADER]: Becoming Leader. State: Replica: 069a1ba2b8f34447abfd770459a342a6, State: Running, Role: LEADER
I20260808 13:09:51.676270 32083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } }
I20260808 13:09:51.676855 32078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:51.679116 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "069a1ba2b8f34447abfd770459a342a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } } }
I20260808 13:09:51.679523 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:51.680855 32085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 069a1ba2b8f34447abfd770459a342a6. Latest consensus state: current_term: 1 leader_uuid: "069a1ba2b8f34447abfd770459a342a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069a1ba2b8f34447abfd770459a342a6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42847 } } }
I20260808 13:09:51.681143 32085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:51.683079 32088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:51.683756 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:51.645395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.31.59.190:42847)
0808 13:09:51.646408 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:51.646424 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:51.647761 (+ 1337us) server_negotiation.cc:407] Connection header received
0808 13:09:51.657000 (+ 9239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:51.657024 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:51.657505 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:51.657917 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:51.660587 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:51.662313 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:51.664715 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:51.665372 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:51.665540 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:51.672719 (+ 7179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:51.672735 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:51.672755 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:51.672793 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:51.676808 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:51.677933 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:51.677944 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:51.677951 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:51.678172 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:51.678816 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:51.678822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:51.678825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:51.679598 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:51.679906 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:51.681548 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0808 13:09:51.681990 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":450,"threads_started":1}
I20260808 13:09:51.690861 32088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:51.693229 32103 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:09:51.701216 32088 catalog_manager.cc:1347] Generated new cluster ID: f5ed2bf4790a4e4a96e966445276abeb
I20260808 13:09:51.701360 32088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:51.717005 32088 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:51.718393 32088 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:51.726970 32088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 069a1ba2b8f34447abfd770459a342a6: Generated new TSK 0
I20260808 13:09:51.728025 32088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:09:51.778628 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:52.222546 32109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:52.222916 32109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:52.223145 32109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:52.233343 32109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:52.233515 32109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:52.233718 32109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:09:52.247155 32109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:52.248683 32109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:52.250547 32109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:52.268106 32115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:52.268293 32118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:52.268515 32116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:52.269761 32109 server_base.cc:1034] running on GCE node
I20260808 13:09:52.270490 32109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:52.271494 32109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:52.272719 32109 hybrid_clock.cc:648] HybridClock initialized: now 1786194592272652 us; error 37 us; skew 500 ppm
I20260808 13:09:52.273048 32109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:52 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194592, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:52.277048 32109 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:09:52.279055 32109 webserver.cc:460] Webserver started at http://127.31.59.129:37667/ using document root <none> and password file <none>
I20260808 13:09:52.279992 32109 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:52.280238 32109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:52.280704 32109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:52.284288 32109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "234d09466aa74b5ca77fe71f4b9885a0"
format_stamp: "Formatted at 2026-08-08 13:09:52 on dist-test-slave-49h3"
server_key: "987f3be448facb41099726d25f93f2f5"
server_key_iv: "9f1f4b043b37d3e4a93949877410b66b"
server_key_version: "encryptionkey@0"
I20260808 13:09:52.285111 32109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "234d09466aa74b5ca77fe71f4b9885a0"
format_stamp: "Formatted at 2026-08-08 13:09:52 on dist-test-slave-49h3"
server_key: "987f3be448facb41099726d25f93f2f5"
server_key_iv: "9f1f4b043b37d3e4a93949877410b66b"
server_key_version: "encryptionkey@0"
I20260808 13:09:52.291419 32109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260808 13:09:52.295815 32126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:52.297082 32109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:52.297254 32109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "234d09466aa74b5ca77fe71f4b9885a0"
format_stamp: "Formatted at 2026-08-08 13:09:52 on dist-test-slave-49h3"
server_key: "987f3be448facb41099726d25f93f2f5"
server_key_iv: "9f1f4b043b37d3e4a93949877410b66b"
server_key_version: "encryptionkey@0"
I20260808 13:09:52.297508 32109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:52.331003 32109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:52.336741 32109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:52.337138 32109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:52.338673 32109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:52.340553 32109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:52.340715 32109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:52.340939 32109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:52.341073 32109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:52.410331 32109 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:38401
I20260808 13:09:52.410351 32249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:38401 every 8 connection(s)
I20260808 13:09:52.411859 32109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:09:52.419628 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32109
I20260808 13:09:52.419906 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:09:52.420595 31982 external_mini_cluster.cc:1428] Setting key b25511ce62d0e16b23bd0cf875b9d8df
Aug 08 13:09:52 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194592, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:52.438891 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:52.415041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:37147 (local address 127.31.59.190:42847)
0808 13:09:52.415735 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:52.415740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:52.417101 (+ 1361us) server_negotiation.cc:407] Connection header received
0808 13:09:52.419724 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:52.419730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:52.419823 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:52.419933 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:52.421927 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:52.423108 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:52.425274 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:52.426370 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:52.426532 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:52.431506 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:52.431512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:52.431515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:52.431548 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:52.435166 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:52.436200 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:52.436203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:52.436206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:52.436331 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:52.436883 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:52.436886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:52.436888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:52.437176 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:52.437297 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:52.438379 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0808 13:09:52.438552 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":567,"thread_start_us":377,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
I20260808 13:09:52.440466 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:52.415893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42847 (local address 127.31.59.129:37147)
0808 13:09:52.416904 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:52.416942 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:52.419437 (+ 2495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:52.420227 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:52.420237 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:52.420917 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:52.421731 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:52.421753 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:52.423254 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:52.423258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:52.425040 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:52.425056 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:52.426778 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:52.426789 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:52.427228 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:52.428045 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:52.428070 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:52.431233 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:52.435319 (+ 4086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:52.435329 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:52.435348 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:52.436094 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:52.436433 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:52.436438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:52.436443 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:52.436790 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:52.437303 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:52.437311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:52.437865 (+ 554us) client_negotiation.cc:769] Sending connection context
0808 13:09:52.438242 (+ 377us) client_negotiation.cc:240] Negotiation successful
0808 13:09:52.438881 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":536,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
I20260808 13:09:52.442428 32250 heartbeater.cc:347] Connected to a master server at 127.31.59.190:42847
I20260808 13:09:52.442830 32250 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:52.443894 32250 heartbeater.cc:511] Master 127.31.59.190:42847 requested a full tablet report, sending...
I20260808 13:09:52.446449 32042 ts_manager.cc:194] Registered new tserver with Master: 234d09466aa74b5ca77fe71f4b9885a0 (127.31.59.129:38401)
I20260808 13:09:52.448094 32042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:37147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:09:52.464636 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:53.033998 32257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:53.034363 32257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:53.034482 32257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:53.043834 32257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:53.044008 32257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:53.044322 32257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:09:53.057412 32257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:53.058924 32257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:53.060591 32257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:53.075493 32264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:53.076411 32267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:53.076462 32265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:53.083212 32257 server_base.cc:1034] running on GCE node
I20260808 13:09:53.083879 32257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:53.084900 32257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:53.086122 32257 hybrid_clock.cc:648] HybridClock initialized: now 1786194593086040 us; error 51 us; skew 500 ppm
I20260808 13:09:53.086453 32257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:53 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194593, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:53.090821 32257 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:09:53.092993 32257 webserver.cc:460] Webserver started at http://127.31.59.130:35543/ using document root <none> and password file <none>
I20260808 13:09:53.093897 32257 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:53.094028 32257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:53.094352 32257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:53.097864 32257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "cd28b8ee031a4087815b30be2d00e915"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "abc0f69970a3b35134fe4969e8387d3c"
server_key_iv: "e71a72da8591dddd60009cd329e64462"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.098763 32257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "cd28b8ee031a4087815b30be2d00e915"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "abc0f69970a3b35134fe4969e8387d3c"
server_key_iv: "e71a72da8591dddd60009cd329e64462"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.104664 32257 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260808 13:09:53.109071 32274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:53.110532 32257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:53.110755 32257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "cd28b8ee031a4087815b30be2d00e915"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "abc0f69970a3b35134fe4969e8387d3c"
server_key_iv: "e71a72da8591dddd60009cd329e64462"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.111079 32257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:53.139981 32257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:53.145764 32257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:53.146271 32257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:53.147589 32257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:53.149760 32257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:53.149871 32257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:53.150028 32257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:53.150107 32257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:53.221141 32257 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:46555
I20260808 13:09:53.221204 32394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:46555 every 8 connection(s)
I20260808 13:09:53.222792 32257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:09:53.231472 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32257
I20260808 13:09:53.231806 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:09:53.232462 31982 external_mini_cluster.cc:1428] Setting key 81eadcb35a89997b1ed46343c2125716
Aug 08 13:09:53 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194593, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:53.251657 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:53.225972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:59975 (local address 127.31.59.190:42847)
0808 13:09:53.226620 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:53.226624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:53.228001 (+ 1377us) server_negotiation.cc:407] Connection header received
0808 13:09:53.231008 (+ 3007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:53.231012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:53.231086 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:53.231177 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:53.233712 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:53.234822 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:53.237647 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:53.238234 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:53.238337 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:53.243411 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:53.243418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:53.243423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:53.243465 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:53.247933 (+ 4468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:53.248826 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:53.248830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:53.248833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:53.249018 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:53.249630 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:53.249635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:53.249638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:53.249996 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:53.250128 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:53.250996 (+ 868us) server_negotiation.cc:299] Negotiation successful
0808 13:09:53.251221 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":449,"threads_started":1}
I20260808 13:09:53.252700 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:53.226482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42847 (local address 127.31.59.130:59975)
0808 13:09:53.227618 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:53.227684 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:53.230703 (+ 3019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:53.231504 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:53.231517 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:53.232365 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:53.233513 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:53.233549 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:53.235031 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:53.235037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:53.237496 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:53.237534 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:53.238608 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:53.238619 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:53.238951 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:53.239771 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:53.239800 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:53.243053 (+ 3253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:53.248077 (+ 5024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:53.248084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:53.248099 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:53.248689 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:53.249131 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:53.249134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:53.249136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:53.249512 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:53.250125 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:53.250134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:53.250525 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 13:09:53.250854 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 13:09:53.251470 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":634,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
I20260808 13:09:53.254292 32395 heartbeater.cc:347] Connected to a master server at 127.31.59.190:42847
I20260808 13:09:53.254591 32395 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:53.255597 32395 heartbeater.cc:511] Master 127.31.59.190:42847 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
I20260808 13:09:53.257148 32042 ts_manager.cc:194] Registered new tserver with Master: cd28b8ee031a4087815b30be2d00e915 (127.31.59.130:46555)
I20260808 13:09:53.258232 32042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:59975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:09:53.277343 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:09:53.452068 32250 heartbeater.cc:503] Master 127.31.59.190:42847 was elected leader, sending a full tablet report...
W20260808 13:09:53.854724 32401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:53.855197 32401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:53.855509 32401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:53.867689 32401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:53.867942 32401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:53.868217 32401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:09:53.883144 32401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:42847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:53.884681 32401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:53.886252 32401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:53.906648 32409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:53.906339 32406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:53.907371 32407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:53.908703 32401 server_base.cc:1034] running on GCE node
I20260808 13:09:53.909583 32401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:53.910827 32401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:53.912124 32401 hybrid_clock.cc:648] HybridClock initialized: now 1786194593911986 us; error 46 us; skew 500 ppm
I20260808 13:09:53.912592 32401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:53 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194593, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:53.917594 32401 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:09:53.919757 32401 webserver.cc:460] Webserver started at http://127.31.59.131:35293/ using document root <none> and password file <none>
I20260808 13:09:53.920763 32401 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:53.921003 32401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:53.921514 32401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:53.925498 32401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "8a7741adf1584452b41227d1994b9a30"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "81e50d21566264abe99b2f4b3956c48d"
server_key_iv: "6ff4560a58e75ad8acda278b97e11c61"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.926563 32401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "8a7741adf1584452b41227d1994b9a30"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "81e50d21566264abe99b2f4b3956c48d"
server_key_iv: "6ff4560a58e75ad8acda278b97e11c61"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.932969 32401 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260808 13:09:53.938788 32417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:53.940754 32401 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260808 13:09:53.941036 32401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8a7741adf1584452b41227d1994b9a30"
format_stamp: "Formatted at 2026-08-08 13:09:53 on dist-test-slave-49h3"
server_key: "81e50d21566264abe99b2f4b3956c48d"
server_key_iv: "6ff4560a58e75ad8acda278b97e11c61"
server_key_version: "encryptionkey@0"
I20260808 13:09:53.941629 32401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:53.974397 32401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:53.981141 32401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:53.981643 32401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:53.983486 32401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:53.985841 32401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:53.985991 32401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:53.986243 32401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:53.986377 32401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:54.083222 32401 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:34917
I20260808 13:09:54.083469 32537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:34917 every 8 connection(s)
I20260808 13:09:54.085009 32401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:09:54.089519 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32401
I20260808 13:09:54.090036 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:09:54.090950 31982 external_mini_cluster.cc:1428] Setting key abcf270b7c484e81c3b10561137ceea7
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194593, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:54.114765 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.088695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39065 (local address 127.31.59.190:42847)
0808 13:09:54.089801 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.089808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.091491 (+ 1683us) server_negotiation.cc:407] Connection header received
0808 13:09:54.095201 (+ 3710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.095207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.095326 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.095426 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.097654 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.098686 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.100678 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.101171 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.101302 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.106237 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.106243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.106248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.106298 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.110972 (+ 4674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.111787 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.111791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.111793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.111989 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.112498 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.112501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.112503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.112821 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.112968 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.113811 (+ 843us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.114037 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":644,"threads_started":1}
I20260808 13:09:54.115935 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.089673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42847 (local address 127.31.59.131:39065)
0808 13:09:54.091334 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:54.091380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:54.094888 (+ 3508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:54.095759 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:54.095771 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:54.096591 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:54.097495 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:54.097518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.098821 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.098826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:54.100556 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:54.100569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.101554 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.101561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:54.101793 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.102697 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:54.102720 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:54.105998 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:54.111093 (+ 5095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:54.111100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:54.111114 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:54.111696 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:54.112069 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:54.112072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:54.112074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:54.112407 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:54.112955 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:54.112962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:54.113316 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 13:09:54.113622 (+ 306us) client_negotiation.cc:240] Negotiation successful
0808 13:09:54.114297 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1351,"thread_start_us":921,"threads_started":1}
I20260808 13:09:54.117548 32538 heartbeater.cc:347] Connected to a master server at 127.31.59.190:42847
I20260808 13:09:54.117970 32538 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:54.119196 32538 heartbeater.cc:511] Master 127.31.59.190:42847 requested a full tablet report, sending...
I20260808 13:09:54.121093 32042 ts_manager.cc:194] Registered new tserver with Master: 8a7741adf1584452b41227d1994b9a30 (127.31.59.131:34917)
I20260808 13:09:54.122191 32042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:39065
I20260808 13:09:54.133268 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:09:54.189352 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.167622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.31.59.129:38401)
0808 13:09:54.168685 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.168694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.168730 (+ 36us) server_negotiation.cc:407] Connection header received
0808 13:09:54.168884 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.168892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.169158 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.169322 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.171191 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.172724 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.174646 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.175661 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.175779 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.181832 (+ 6053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.181845 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.181859 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.181893 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.185945 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.186626 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.186633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.186635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.186799 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.187292 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.187295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.187297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.187729 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.188225 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.188407 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.188743 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":612,"threads_started":1}
W20260808 13:09:54.190966 32155 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37184
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:09:54.243112 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.223702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.31.59.129:38401)
0808 13:09:54.223877 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.223884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.224498 (+ 614us) server_negotiation.cc:407] Connection header received
0808 13:09:54.224672 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.224676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.224774 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.224862 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.227539 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.228249 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.230196 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.230637 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.230753 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.236570 (+ 5817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.236576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.236579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.236616 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.239775 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.240592 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.240596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.240598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.240822 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.241225 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.241229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.241233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.241692 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.242005 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.242352 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.242617 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:09:54.245002 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32109
I20260808 13:09:54.260811 32395 heartbeater.cc:503] Master 127.31.59.190:42847 was elected leader, sending a full tablet report...
I20260808 13:09:54.263103 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.246512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.31.59.129:38401)
0808 13:09:54.246693 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.246698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.246716 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:09:54.246937 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.246941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.246997 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.247072 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.248962 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.249856 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.251757 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.252418 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.252531 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.255977 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.255983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.255986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.256018 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.259691 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.260646 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.260651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.260653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.260918 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.261483 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.261488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.261491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.261986 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.262297 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.262449 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.262664 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:09:54.264148 32244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:54.349272 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32109
I20260808 13:09:54.365834 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32257
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:09:54.383122 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.366703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.31.59.130:46555)
0808 13:09:54.367482 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.367490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.367515 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:09:54.367578 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.367584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.367802 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.367945 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.369015 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.370086 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.371253 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.371815 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.371908 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.377307 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.377319 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.377333 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.377367 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.380175 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.380968 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.380975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.380977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.381092 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.381720 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.381723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.381725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.382064 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.382290 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.382622 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.382831 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":328,"threads_started":1}
I20260808 13:09:54.384577 32390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:54.468735 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32257
I20260808 13:09:54.485572 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32401
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[31991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:09:54.501554 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:54.486616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.31.59.131:34917)
0808 13:09:54.487172 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:54.487179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:54.487210 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:09:54.487274 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:54.487280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:54.487492 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:54.487624 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:54.489158 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.490291 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.491622 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:54.492177 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:54.492275 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:54.496060 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:54.496072 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:54.496086 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:54.496122 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:54.498920 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.499534 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.499541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.499543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.499662 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:54.500000 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:54.500004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:54.500006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:54.500346 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:54.500638 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:54.500927 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 13:09:54.501190 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":248,"threads_started":1}
I20260808 13:09:54.502760 32533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:54.583385 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32401
I20260808 13:09:54.600490 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32004
I20260808 13:09:54.601718 32072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:54.703799 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32004
2026-08-08T13:09:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4769 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[32561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[32561](info): set up 2 sockets
Aug 08 13:09:54 dist-test-slave-49h3 krb5kdc[32561](info): commencing operation
krb5kdc: starting...
W20260808 13:09:55.741627 31982 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:09:55 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:09:55.924943 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45979
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:56.246371 32574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:56.246675 32574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:56.246807 32574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:56.254108 32574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:56.254222 32574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:56.254299 32574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:56.254382 32574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:56.254447 32574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:56.264430 32574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:56.265832 32574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:56.267194 32574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:56.278839 32582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:56.278877 32580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:56.278839 32579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:56.279577 32574 server_base.cc:1034] running on GCE node
I20260808 13:09:56.280347 32574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:56.281804 32574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:56.283013 32574 hybrid_clock.cc:648] HybridClock initialized: now 1786194596282962 us; error 39 us; skew 500 ppm
I20260808 13:09:56.283346 32574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:56 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194596, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:56.289318 32574 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:09:56.290789 32574 webserver.cc:460] Webserver started at http://127.31.59.190:44619/ using document root <none> and password file <none>
I20260808 13:09:56.291442 32574 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:56.291550 32574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:56.291821 32574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:56.294917 32574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "9e44a3d75066425e97654504acdb6e04"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "260b40fe2a14fcf0c03b217f353fe81f"
server_key_iv: "c4a18edc0178dd25577c6a01f5c3195e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.295640 32574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "9e44a3d75066425e97654504acdb6e04"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "260b40fe2a14fcf0c03b217f353fe81f"
server_key_iv: "c4a18edc0178dd25577c6a01f5c3195e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.300763 32574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 13:09:56.304085 32589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:56.305332 32574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:56.305516 32574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "9e44a3d75066425e97654504acdb6e04"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "260b40fe2a14fcf0c03b217f353fe81f"
server_key_iv: "c4a18edc0178dd25577c6a01f5c3195e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.305688 32574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:56.340665 32574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:56.345006 32574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:56.345268 32574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:56.362776 32574 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:45979
I20260808 13:09:56.362802 32647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:45979 every 8 connection(s)
I20260808 13:09:56.364151 32574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:56.368042 32648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:56.370960 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32574
I20260808 13:09:56.371199 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:09:56.371968 31982 external_mini_cluster.cc:1428] Setting key 0c216ad4003ed6daea110b551f15c235
I20260808 13:09:56.380519 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04: Bootstrap starting.
I20260808 13:09:56.384256 32648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:56.385591 32648 log.cc:824] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:09:56 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:56.388849 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04: No bootstrap required, opened a new log
I20260808 13:09:56.396343 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:56.374099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.31.59.190:45979)
0808 13:09:56.374878 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:56.374889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:56.374930 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:09:56.376585 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:56.376607 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:56.377009 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:56.377326 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:56.379110 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.380393 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.382110 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.382801 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.382939 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:56.387650 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:56.387662 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:56.387680 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:56.387728 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:56.391599 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:56.392247 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:56.392255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:56.392268 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:56.392476 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:56.392826 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:56.392831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:56.392835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:56.393578 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:56.393873 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:56.394468 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 13:09:56.394930 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":349,"threads_started":1}
I20260808 13:09:56.399003 32648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } }
I20260808 13:09:56.399430 32648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:56.399507 32648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e44a3d75066425e97654504acdb6e04, State: Initialized, Role: FOLLOWER
I20260808 13:09:56.399958 32648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } }
I20260808 13:09:56.400156 32648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:56.400255 32648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:56.400367 32648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:56.402457 32648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } }
I20260808 13:09:56.402932 32648 leader_election.cc:302] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e44a3d75066425e97654504acdb6e04; no voters:
I20260808 13:09:56.403573 32648 leader_election.cc:288] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:56.403715 32653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:56.404661 32653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [term 1 LEADER]: Becoming Leader. State: Replica: 9e44a3d75066425e97654504acdb6e04, State: Running, Role: LEADER
I20260808 13:09:56.405143 32653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } }
I20260808 13:09:56.405611 32648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:56.407703 32655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e44a3d75066425e97654504acdb6e04. Latest consensus state: current_term: 1 leader_uuid: "9e44a3d75066425e97654504acdb6e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } } }
I20260808 13:09:56.407682 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e44a3d75066425e97654504acdb6e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e44a3d75066425e97654504acdb6e04" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45979 } } }
I20260808 13:09:56.408093 32655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:56.408190 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:56.409678 32658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:56.413853 32658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:56.421940 32658 catalog_manager.cc:1347] Generated new cluster ID: 934beb75a52c4d179fa7a881f72a3ebe
I20260808 13:09:56.422060 32658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:56.433949 32658 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:56.435228 32658 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:56.455495 32658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e44a3d75066425e97654504acdb6e04: Generated new TSK 0
I20260808 13:09:56.456394 32658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:09:56.504387 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:56.823344 32675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:56.823616 32675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:56.823721 32675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:56.830909 32675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:56.831015 32675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:56.831143 32675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:09:56.840765 32675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:56.842140 32675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:56.843444 32675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:56.856169 32680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:56.856199 32681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:56.856220 32684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:56.856918 32675 server_base.cc:1034] running on GCE node
I20260808 13:09:56.857563 32675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:56.858474 32675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:56.859683 32675 hybrid_clock.cc:648] HybridClock initialized: now 1786194596859639 us; error 33 us; skew 500 ppm
I20260808 13:09:56.859983 32675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:56 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194596, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:56.863090 32675 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:09:56.864843 32675 webserver.cc:460] Webserver started at http://127.31.59.129:33013/ using document root <none> and password file <none>
I20260808 13:09:56.865667 32675 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:56.865813 32675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:56.866196 32675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:56.869637 32675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "2a9018ad73df4acfbe42512da0c32039"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "51e191ee395397186c97eb883721224d"
server_key_iv: "20a8407d05a0b92fdc726530cda3172e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.870404 32675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "2a9018ad73df4acfbe42512da0c32039"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "51e191ee395397186c97eb883721224d"
server_key_iv: "20a8407d05a0b92fdc726530cda3172e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.875597 32675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:56.879030 32691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:56.880332 32675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 13:09:56.880473 32675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "2a9018ad73df4acfbe42512da0c32039"
format_stamp: "Formatted at 2026-08-08 13:09:56 on dist-test-slave-49h3"
server_key: "51e191ee395397186c97eb883721224d"
server_key_iv: "20a8407d05a0b92fdc726530cda3172e"
server_key_version: "encryptionkey@0"
I20260808 13:09:56.880671 32675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:56.893855 32675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:56.898180 32675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:56.898437 32675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:56.899358 32675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:56.900655 32675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:56.900727 32675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:56.900837 32675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:56.900899 32675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:56.931011 32675 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:39869
I20260808 13:09:56.931057 342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:39869 every 8 connection(s)
I20260808 13:09:56.932348 32675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:09:56.942924 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 32675
I20260808 13:09:56.943169 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:09:56.943946 31982 external_mini_cluster.cc:1428] Setting key 7bcbbbc41379bd3246bdc1a21d0b0867
Aug 08 13:09:56 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194596, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:56.951861 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:56.934774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:58323 (local address 127.31.59.190:45979)
0808 13:09:56.935190 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:56.935195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:56.936165 (+ 970us) server_negotiation.cc:407] Connection header received
0808 13:09:56.937957 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:56.937963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:56.938038 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:56.938145 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:56.939567 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.940323 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.941862 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.942274 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.942368 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:56.946339 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:56.946344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:56.946347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:56.946380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:56.948846 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:56.949704 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:56.949708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:56.949711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:56.949830 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:56.950252 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:56.950255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:56.950257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:56.950524 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:56.950632 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:56.951397 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 13:09:56.951563 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":190,"threads_started":1}
I20260808 13:09:56.952965 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:56.935298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45979 (local address 127.31.59.129:58323)
0808 13:09:56.935978 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:56.936024 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:56.937699 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:56.938322 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:56.938330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:56.938859 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:56.939426 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:56.939443 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.940438 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.940442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:56.941742 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:56.941765 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:56.942505 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:56.942513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:56.942815 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:56.943548 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:56.943580 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:56.946089 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:56.949004 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:56.949013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:56.949032 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:56.949601 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:56.949921 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:56.949925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:56.949928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:56.950160 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:56.950633 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:56.950640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:56.951061 (+ 421us) client_negotiation.cc:769] Sending connection context
0808 13:09:56.951331 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 13:09:56.951859 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":267,"threads_started":1}
I20260808 13:09:56.954445 344 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45979
I20260808 13:09:56.954770 344 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:56.955677 344 heartbeater.cc:511] Master 127.31.59.190:45979 requested a full tablet report, sending...
I20260808 13:09:56.957938 32606 ts_manager.cc:194] Registered new tserver with Master: 2a9018ad73df4acfbe42512da0c32039 (127.31.59.129:39869)
I20260808 13:09:56.959452 32606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:58323
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:09:56.987025 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:57.385222 350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:57.385529 350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:57.385680 350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:57.392972 350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:57.393096 350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:57.393253 350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:09:57.403420 350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:57.404752 350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:57.406208 350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:57.419866 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:57.419905 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:57.420107 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:57.420684 350 server_base.cc:1034] running on GCE node
I20260808 13:09:57.421243 350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:57.422221 350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:57.423420 350 hybrid_clock.cc:648] HybridClock initialized: now 1786194597423376 us; error 31 us; skew 500 ppm
I20260808 13:09:57.423733 350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:57 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:57.426797 350 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:09:57.428210 350 webserver.cc:460] Webserver started at http://127.31.59.130:33725/ using document root <none> and password file <none>
I20260808 13:09:57.428862 350 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:57.428978 350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:57.429277 350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:57.432447 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "6e09dbbd3315449f9d181717e27be04a"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "047977dca4c69359c5370d1a7edfae93"
server_key_iv: "ce1fbd73fcd6032c755e306e27efd349"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.433271 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "6e09dbbd3315449f9d181717e27be04a"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "047977dca4c69359c5370d1a7edfae93"
server_key_iv: "ce1fbd73fcd6032c755e306e27efd349"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.438333 350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 13:09:57.441612 365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.442888 350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:09:57.443033 350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "6e09dbbd3315449f9d181717e27be04a"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "047977dca4c69359c5370d1a7edfae93"
server_key_iv: "ce1fbd73fcd6032c755e306e27efd349"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.443214 350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:57.461589 350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:57.465471 350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:57.465770 350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:57.466771 350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:57.468206 350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:57.468318 350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.468447 350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:57.468501 350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.503211 350 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44231
I20260808 13:09:57.503268 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44231 every 8 connection(s)
I20260808 13:09:57.504503 350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:09:57.507431 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 350
I20260808 13:09:57.507637 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:09:57.508070 31982 external_mini_cluster.cc:1428] Setting key 2e535df68eecb973ef1d273054f584b9
Aug 08 13:09:57 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:57.524235 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:57.507099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:47353 (local address 127.31.59.190:45979)
0808 13:09:57.507611 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:57.507619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:57.508671 (+ 1052us) server_negotiation.cc:407] Connection header received
0808 13:09:57.510767 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:57.510772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:57.510850 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:57.510941 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:57.512413 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.513137 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.514667 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.515069 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.515176 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:57.518709 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:57.518714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:57.518718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:57.518758 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:57.521175 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:57.521971 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:57.521975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:57.521978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:57.522100 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:57.522513 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:57.522516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:57.522519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:57.522783 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:57.522924 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:57.523597 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 13:09:57.523832 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
I20260808 13:09:57.525158 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:57.507671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45979 (local address 127.31.59.130:47353)
0808 13:09:57.508474 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:57.508542 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:57.510510 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:57.511200 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:57.511208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:57.511700 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:57.512247 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:57.512265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.513286 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.513291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:57.514531 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:57.514543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.515292 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.515299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:57.515536 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:57.516178 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:57.516204 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:57.518502 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:57.521309 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:57.521318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:57.521335 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:57.521853 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:57.522200 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:57.522205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:57.522209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:57.522399 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:57.522905 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:57.522912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:57.523242 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 13:09:57.523486 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 13:09:57.523998 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":310,"threads_started":1}
I20260808 13:09:57.526657 488 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45979
I20260808 13:09:57.526974 488 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:57.527769 488 heartbeater.cc:511] Master 127.31.59.190:45979 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
I20260808 13:09:57.529408 32606 ts_manager.cc:194] Registered new tserver with Master: 6e09dbbd3315449f9d181717e27be04a (127.31.59.130:44231)
I20260808 13:09:57.530458 32606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:47353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:09:57.550407 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:57.870059 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:57.870679 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:57.870848 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:57.878455 494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:57.878568 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:57.878687 494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:09:57.888792 494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:57.890276 494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:57.891599 494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:57.904065 499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:57.904057 502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:57.904111 500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:57.905463 494 server_base.cc:1034] running on GCE node
I20260808 13:09:57.906082 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:57.907037 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:57.908228 494 hybrid_clock.cc:648] HybridClock initialized: now 1786194597908179 us; error 47 us; skew 500 ppm
I20260808 13:09:57.908524 494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:57 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:57.911489 494 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:09:57.912844 494 webserver.cc:460] Webserver started at http://127.31.59.131:38207/ using document root <none> and password file <none>
I20260808 13:09:57.913522 494 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:57.913642 494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:57.913939 494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:57.916704 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "41094c655dfc43ff97071d57b67d9be3"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "1586a5666c4d9fe97bb3d3d579da0e4e"
server_key_iv: "3d4e0bbec85b864b2cea29fb23b6b6c2"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.917483 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "41094c655dfc43ff97071d57b67d9be3"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "1586a5666c4d9fe97bb3d3d579da0e4e"
server_key_iv: "3d4e0bbec85b864b2cea29fb23b6b6c2"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.922364 494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:09:57.925995 509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.927217 494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:09:57.927374 494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "41094c655dfc43ff97071d57b67d9be3"
format_stamp: "Formatted at 2026-08-08 13:09:57 on dist-test-slave-49h3"
server_key: "1586a5666c4d9fe97bb3d3d579da0e4e"
server_key_iv: "3d4e0bbec85b864b2cea29fb23b6b6c2"
server_key_version: "encryptionkey@0"
I20260808 13:09:57.927549 494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:57.944010 494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:57.947964 494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:57.948230 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:57.949162 494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:09:57.950516 494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:09:57.950594 494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.950719 494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:09:57.950778 494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:57.963027 344 heartbeater.cc:503] Master 127.31.59.190:45979 was elected leader, sending a full tablet report...
I20260808 13:09:57.980821 494 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:37115
I20260808 13:09:57.980873 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:37115 every 8 connection(s)
I20260808 13:09:57.982182 494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:09:57.989568 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 494
I20260808 13:09:57.989787 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:09:57.990218 31982 external_mini_cluster.cc:1428] Setting key 3fac8f4c4667b5c35199f9ff53f02464
Aug 08 13:09:57 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:09:58.001885 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:57.984654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:41705 (local address 127.31.59.190:45979)
0808 13:09:57.984848 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:57.984853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:57.986003 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 13:09:57.987780 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:57.987786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:57.987869 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:57.987959 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:57.989565 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.990457 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.992443 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.993057 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.993190 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:57.996585 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:57.996591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:57.996595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:57.996629 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:57.998958 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:57.999650 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:57.999655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:57.999657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:57.999789 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.000203 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.000206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.000208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.000484 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.000616 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.001295 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.001552 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:09:58.002835 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:57.985090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45979 (local address 127.31.59.131:41705)
0808 13:09:57.985845 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:57.985888 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:57.987511 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:57.988194 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:57.988203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:57.988688 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:57.989343 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:57.989369 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.990616 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.990621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:57.992304 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:57.992319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:57.993278 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:57.993283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:57.993586 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:57.994211 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:57.994242 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:57.996310 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:57.999090 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:57.999099 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:57.999115 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:57.999526 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:57.999890 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:57.999896 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:57.999900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:58.000089 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:58.000606 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:58.000614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:58.000944 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.001180 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.001742 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":326,"threads_started":1}
I20260808 13:09:58.004189 631 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45979
I20260808 13:09:58.004489 631 heartbeater.cc:464] Registering TS with master...
I20260808 13:09:58.005285 631 heartbeater.cc:511] Master 127.31.59.190:45979 requested a full tablet report, sending...
I20260808 13:09:58.006762 32606 ts_manager.cc:194] Registered new tserver with Master: 41094c655dfc43ff97071d57b67d9be3 (127.31.59.131:37115)
I20260808 13:09:58.007620 32606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:41705
I20260808 13:09:58.017951 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:09:58.036464 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.024114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.31.59.190:45979)
0808 13:09:58.024278 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.024283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.024741 (+ 458us) server_negotiation.cc:407] Connection header received
0808 13:09:58.024913 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.024918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.025005 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.025089 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.026624 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.027424 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.028663 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.029115 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.029226 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.031504 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.031508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.031512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.031546 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.033919 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.034528 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.034533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.034537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.034669 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.035018 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.035022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.035024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.035332 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.035580 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.035927 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.036161 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:09:58.042915 32606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:09:58.049991 32606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:09:58.072788 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.061685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44231 (local address 127.0.0.1:60082)
0808 13:09:58.063135 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:58.063188 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:58.063374 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:58.064532 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:58.064540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:58.064580 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:09:58.064986 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.064998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.067026 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.067032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.069498 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.069517 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.071742 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.071749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.071996 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.072053 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.072181 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.072389 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"threads_started":1}
I20260808 13:09:58.073906 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.061927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.31.59.130:44231)
0808 13:09:58.063681 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.063690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.063719 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:09:58.063821 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.063828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.064061 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.064269 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:09:58.065175 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.066848 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.069706 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.071224 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.071370 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.071584 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.073359 (+ 1775us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.073570 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":490,"threads_started":1}
I20260808 13:09:58.079579 416 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab876f512da47bc8bed7a4b77d8fb90 (DEFAULT_TABLE table=test-table [id=041e93ec11b34df590acb4cf92c7314a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:09:58.081096 416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab876f512da47bc8bed7a4b77d8fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:58.081663 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.062017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:39869 (local address 127.0.0.1:57298)
0808 13:09:58.073048 (+ 11031us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:58.073068 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:58.073174 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:58.075575 (+ 2401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:58.075581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:58.075612 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:09:58.075798 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.075808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.077322 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.077327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.079171 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.079187 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.080939 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.080944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.081112 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.081143 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.081220 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.081346 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10909,"thread_start_us":715,"threads_started":1}
I20260808 13:09:58.082520 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.062688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.31.59.129:39869)
0808 13:09:58.074633 (+ 11945us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.074643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.074676 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:09:58.074781 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.074791 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.075072 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.075293 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:09:58.075983 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.077169 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.079354 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.080786 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.080925 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.081986 (+ 1061us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.082056 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.082217 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11772,"thread_start_us":11618,"threads_started":1}
I20260808 13:09:58.083405 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.063548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37115 (local address 127.0.0.1:38528)
0808 13:09:58.073948 (+ 10400us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:58.073966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:58.074078 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:58.076696 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:58.076700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:58.076726 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:09:58.077128 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.077140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.079384 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.079389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.080678 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.080690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.082651 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.082657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.082834 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.082864 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.082944 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.083070 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10263,"thread_start_us":10602,"threads_started":1}
I20260808 13:09:58.084468 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.063871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.31.59.131:37115)
0808 13:09:58.076124 (+ 12253us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.076132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.076162 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:09:58.076250 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.076257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.076454 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.077389 (+ 935us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:09:58.077685 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.079218 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.080830 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.082462 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.082725 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.082948 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.083013 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.083154 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12085,"thread_start_us":11932,"threads_started":1}
I20260808 13:09:58.086162 32741 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab876f512da47bc8bed7a4b77d8fb90 (DEFAULT_TABLE table=test-table [id=041e93ec11b34df590acb4cf92c7314a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:09:58.087675 32741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab876f512da47bc8bed7a4b77d8fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:58.088241 561 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab876f512da47bc8bed7a4b77d8fb90 (DEFAULT_TABLE table=test-table [id=041e93ec11b34df590acb4cf92c7314a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:09:58.089672 561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab876f512da47bc8bed7a4b77d8fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:58.093082 651 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Bootstrap starting.
I20260808 13:09:58.100414 651 tablet_bootstrap.cc:654] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:58.103112 651 log.cc:824] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:58.105402 652 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Bootstrap starting.
I20260808 13:09:58.108193 654 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Bootstrap starting.
I20260808 13:09:58.111392 652 tablet_bootstrap.cc:654] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:58.112371 651 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: No bootstrap required, opened a new log
I20260808 13:09:58.112759 651 ts_tablet_manager.cc:1397] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.011s
I20260808 13:09:58.113710 652 log.cc:824] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:58.114629 654 tablet_bootstrap.cc:654] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:58.116954 654 log.cc:824] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:58.119794 652 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: No bootstrap required, opened a new log
I20260808 13:09:58.120258 652 ts_tablet_manager.cc:1397] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260808 13:09:58.121902 654 tablet_bootstrap.cc:492] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: No bootstrap required, opened a new log
I20260808 13:09:58.122275 654 ts_tablet_manager.cc:1397] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260808 13:09:58.132516 651 raft_consensus.cc:348] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.133241 651 raft_consensus.cc:374] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:58.133392 651 raft_consensus.cc:729] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e09dbbd3315449f9d181717e27be04a, State: Initialized, Role: FOLLOWER
I20260808 13:09:58.134202 651 consensus_queue.cc:260] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.135882 488 heartbeater.cc:503] Master 127.31.59.190:45979 was elected leader, sending a full tablet report...
I20260808 13:09:58.136649 651 ts_tablet_manager.cc:1428] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Time spent starting tablet: real 0.023s user 0.010s sys 0.009s
I20260808 13:09:58.138469 652 raft_consensus.cc:348] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.138988 652 raft_consensus.cc:374] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:58.139113 652 raft_consensus.cc:729] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41094c655dfc43ff97071d57b67d9be3, State: Initialized, Role: FOLLOWER
I20260808 13:09:58.139732 652 consensus_queue.cc:260] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.141110 631 heartbeater.cc:503] Master 127.31.59.190:45979 was elected leader, sending a full tablet report...
I20260808 13:09:58.141939 652 ts_tablet_manager.cc:1428] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Time spent starting tablet: real 0.021s user 0.003s sys 0.016s
I20260808 13:09:58.142495 654 raft_consensus.cc:348] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.143234 654 raft_consensus.cc:374] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:58.143422 654 raft_consensus.cc:729] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9018ad73df4acfbe42512da0c32039, State: Initialized, Role: FOLLOWER
I20260808 13:09:58.144191 654 consensus_queue.cc:260] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.146415 654 ts_tablet_manager.cc:1428] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Time spent starting tablet: real 0.024s user 0.016s sys 0.005s
W20260808 13:09:58.184687 345 tablet.cc:2369] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:09:58.233768 633 tablet.cc:2369] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:09:58.256520 489 tablet.cc:2369] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:09:58.311864 661 raft_consensus.cc:482] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:09:58.312170 661 raft_consensus.cc:504] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.314077 661 leader_election.cc:288] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e09dbbd3315449f9d181717e27be04a (127.31.59.130:44231), 41094c655dfc43ff97071d57b67d9be3 (127.31.59.131:37115)
I20260808 13:09:58.323124 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.314570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:36569 (local address 127.31.59.131:37115)
0808 13:09:58.314836 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.314842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.316087 (+ 1245us) server_negotiation.cc:407] Connection header received
0808 13:09:58.316284 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.316289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.316370 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.316467 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:09:58.317538 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.318337 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.321204 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.322104 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.322275 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.322352 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.322703 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.322834 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260808 13:09:58.323513 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.314967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37115 (local address 127.31.59.129:36569)
0808 13:09:58.315986 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:58.316000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:58.316150 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:58.316481 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:58.316484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:58.316507 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:09:58.317364 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.317376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.318484 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.318488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.321031 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.321043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.322338 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.322345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.322555 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.322580 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.322665 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.322791 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":553,"threads_started":1}
I20260808 13:09:58.324245 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.314597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:41033 (local address 127.31.59.130:44231)
0808 13:09:58.314873 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.314880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.315406 (+ 526us) server_negotiation.cc:407] Connection header received
0808 13:09:58.315603 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.315607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.315669 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.315742 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:09:58.316898 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.317752 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.321323 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.322843 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.323041 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.323121 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.323839 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.323990 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:58.324692 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.314218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44231 (local address 127.31.59.129:41033)
0808 13:09:58.315296 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:58.315311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:58.315467 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:58.315799 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:58.315803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:58.315838 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:09:58.316752 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.316765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.317875 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.317881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.321116 (+ 3235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:58.321130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.323333 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.323341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:58.323606 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.323653 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 13:09:58.323743 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 13:09:58.323870 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":2,"thread_start_us":719,"threads_started":1}
I20260808 13:09:58.325071 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab876f512da47bc8bed7a4b77d8fb90" candidate_uuid: "2a9018ad73df4acfbe42512da0c32039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41094c655dfc43ff97071d57b67d9be3" is_pre_election: true
I20260808 13:09:58.325929 582 raft_consensus.cc:2454] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9018ad73df4acfbe42512da0c32039 in term 0.
I20260808 13:09:58.325939 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab876f512da47bc8bed7a4b77d8fb90" candidate_uuid: "2a9018ad73df4acfbe42512da0c32039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e09dbbd3315449f9d181717e27be04a" is_pre_election: true
I20260808 13:09:58.326853 437 raft_consensus.cc:2454] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9018ad73df4acfbe42512da0c32039 in term 0.
I20260808 13:09:58.327010 32693 leader_election.cc:302] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9018ad73df4acfbe42512da0c32039, 41094c655dfc43ff97071d57b67d9be3; no voters:
I20260808 13:09:58.327509 661 raft_consensus.cc:2781] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:09:58.327824 661 raft_consensus.cc:482] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:09:58.328009 661 raft_consensus.cc:3037] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:58.331681 661 raft_consensus.cc:504] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.332605 661 leader_election.cc:288] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [CANDIDATE]: Term 1 election: Requested vote from peers 6e09dbbd3315449f9d181717e27be04a (127.31.59.130:44231), 41094c655dfc43ff97071d57b67d9be3 (127.31.59.131:37115)
I20260808 13:09:58.333087 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab876f512da47bc8bed7a4b77d8fb90" candidate_uuid: "2a9018ad73df4acfbe42512da0c32039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e09dbbd3315449f9d181717e27be04a"
I20260808 13:09:58.333243 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab876f512da47bc8bed7a4b77d8fb90" candidate_uuid: "2a9018ad73df4acfbe42512da0c32039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41094c655dfc43ff97071d57b67d9be3"
I20260808 13:09:58.333300 437 raft_consensus.cc:3037] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:58.333505 582 raft_consensus.cc:3037] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:58.335980 437 raft_consensus.cc:2454] T 0ab876f512da47bc8bed7a4b77d8fb90 P 6e09dbbd3315449f9d181717e27be04a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9018ad73df4acfbe42512da0c32039 in term 1.
I20260808 13:09:58.336429 32692 leader_election.cc:302] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9018ad73df4acfbe42512da0c32039, 6e09dbbd3315449f9d181717e27be04a; no voters:
I20260808 13:09:58.336799 661 raft_consensus.cc:2781] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:58.337551 582 raft_consensus.cc:2454] T 0ab876f512da47bc8bed7a4b77d8fb90 P 41094c655dfc43ff97071d57b67d9be3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9018ad73df4acfbe42512da0c32039 in term 1.
I20260808 13:09:58.337962 661 raft_consensus.cc:686] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [term 1 LEADER]: Becoming Leader. State: Replica: 2a9018ad73df4acfbe42512da0c32039, State: Running, Role: LEADER
I20260808 13:09:58.338667 661 consensus_queue.cc:237] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } }
I20260808 13:09:58.344218 32606 catalog_manager.cc:5495] T 0ab876f512da47bc8bed7a4b77d8fb90 P 2a9018ad73df4acfbe42512da0c32039 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9018ad73df4acfbe42512da0c32039 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "2a9018ad73df4acfbe42512da0c32039" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09dbbd3315449f9d181717e27be04a" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a9018ad73df4acfbe42512da0c32039" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41094c655dfc43ff97071d57b67d9be3" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37115 } health_report { overall_health: UNKNOWN } } }
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:09:58.396413 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.372312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.31.59.129:39869)
0808 13:09:58.372627 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.372634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.374062 (+ 1428us) server_negotiation.cc:407] Connection header received
0808 13:09:58.374256 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.374260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.374330 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.374443 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.378384 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.379451 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.382264 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.383017 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.383118 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.388730 (+ 5612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.388757 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.388774 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.388812 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.393159 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.394030 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.394038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.394041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.394296 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.394703 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.394706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.394708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.395165 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.395521 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.395686 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.395951 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:09:58.453533 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.434703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.31.59.129:39869)
0808 13:09:58.434898 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.434905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.435676 (+ 771us) server_negotiation.cc:407] Connection header received
0808 13:09:58.435900 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.435904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.435972 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.436057 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.438643 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.439798 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.441617 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.442426 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.442542 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.447442 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.447449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.447452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.447487 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.451227 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.451797 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.451801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.451803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.451973 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.452297 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.452300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.452302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.452625 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.452892 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.452990 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.453165 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:09:58.509636 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.488836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.31.59.129:39869)
0808 13:09:58.489040 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.489048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.489702 (+ 654us) server_negotiation.cc:407] Connection header received
0808 13:09:58.489891 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.489896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.490017 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.490108 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.492602 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.493962 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.495510 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.496251 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.496367 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.502294 (+ 5927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.502300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.502303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.502341 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.506262 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.507159 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.507164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.507166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.507402 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.507808 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.507812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.507814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.508270 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.508561 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.508814 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.509049 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 13:09:58.512733 32721 tablet_service.cc:2908] Rejecting scan request for tablet 0ab876f512da47bc8bed7a4b77d8fb90: Uninitialized: clean time has not yet been initialized
I20260808 13:09:58.517081 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32675
I20260808 13:09:58.534672 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.518378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.31.59.129:39869)
0808 13:09:58.518551 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.518556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.518576 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:09:58.518730 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.518736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.518851 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.518955 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.520715 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.521663 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.523275 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.524019 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.524116 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.527403 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.527408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.527411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.527443 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.531339 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.532332 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.532336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.532339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.532530 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.532876 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.532879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.532881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.533296 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.533653 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.533875 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.534173 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:09:58.535722 338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:58.628661 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32675
I20260808 13:09:58.647969 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 350
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:09:58.663725 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.648710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.31.59.130:44231)
0808 13:09:58.648897 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.648902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.648923 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:09:58.649156 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.649161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.649236 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.649327 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.650696 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.651418 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.653000 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.653400 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.653562 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.657405 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.657420 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.657436 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.657502 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.660453 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.661080 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.661087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.661090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.661326 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.661966 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.661970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.661972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.662538 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.662920 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.663088 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.663302 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:09:58.664352 483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:58.751099 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 350
I20260808 13:09:58.767143 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 494
Aug 08 13:09:58 dist-test-slave-49h3 krb5kdc[32561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:09:58.781236 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:58.767972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.31.59.131:37115)
0808 13:09:58.768158 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:58.768163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:58.768184 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:09:58.768338 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:58.768342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:58.768408 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:58.768497 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:58.769791 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.770523 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.771847 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:58.772260 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:58.772376 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:58.776429 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:58.776445 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:58.776461 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:58.776498 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:58.779019 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.779619 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.779625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.779628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.779771 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:58.780036 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:58.780039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:58.780041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:58.780379 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:58.780587 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:58.780808 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 13:09:58.780976 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:58.781898 625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:58.868155 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 494
I20260808 13:09:58.884179 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 32574
I20260808 13:09:58.885310 32643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:58.993855 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 32574
2026-08-08T13:09:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4292 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:59 dist-test-slave-49h3 krb5kdc[685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:59 dist-test-slave-49h3 krb5kdc[685](info): set up 2 sockets
Aug 08 13:09:59 dist-test-slave-49h3 krb5kdc[685](info): commencing operation
krb5kdc: starting...
W20260808 13:10:00.038005 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:00 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:00.232203 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:46777
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:46777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:00.546813 700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:00.547125 700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:00.547278 700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:00.554440 700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:00.554564 700 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:10:00.554634 700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:00.554720 700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:00.554787 700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:00.554870 700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:00.564677 700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:46777
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:46777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:00.566063 700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:00.567389 700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:00.578837 706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:00.578787 709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:00.578866 707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:00.579383 700 server_base.cc:1034] running on GCE node
I20260808 13:10:00.580212 700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:00.581573 700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:00.582789 700 hybrid_clock.cc:648] HybridClock initialized: now 1786194600582723 us; error 43 us; skew 500 ppm
I20260808 13:10:00.583092 700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:00 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194600, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:00.586133 700 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:00.587524 700 webserver.cc:460] Webserver started at http://127.31.59.190:40621/ using document root <none> and password file <none>
I20260808 13:10:00.588259 700 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:00.588356 700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:00.588660 700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:00.591583 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "1d50715ab5cf4363ba3b8cc466daa628"
format_stamp: "Formatted at 2026-08-08 13:10:00 on dist-test-slave-49h3"
server_key: "e0c8ca1dda5615043af79c44e62954fa"
server_key_iv: "4a0fda23c9355b0297d2d828fede9472"
server_key_version: "encryptionkey@0"
I20260808 13:10:00.592316 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "1d50715ab5cf4363ba3b8cc466daa628"
format_stamp: "Formatted at 2026-08-08 13:10:00 on dist-test-slave-49h3"
server_key: "e0c8ca1dda5615043af79c44e62954fa"
server_key_iv: "4a0fda23c9355b0297d2d828fede9472"
server_key_version: "encryptionkey@0"
I20260808 13:10:00.597174 700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 13:10:00.600404 716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:00.601679 700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:00.601835 700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "1d50715ab5cf4363ba3b8cc466daa628"
format_stamp: "Formatted at 2026-08-08 13:10:00 on dist-test-slave-49h3"
server_key: "e0c8ca1dda5615043af79c44e62954fa"
server_key_iv: "4a0fda23c9355b0297d2d828fede9472"
server_key_version: "encryptionkey@0"
I20260808 13:10:00.602025 700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:00.622411 700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:00.626717 700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:00.626958 700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:00.647258 700 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:46777
I20260808 13:10:00.647266 782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:46777 every 8 connection(s)
I20260808 13:10:00.648613 700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:00.649367 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 700
I20260808 13:10:00.649643 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:00.650373 31982 external_mini_cluster.cc:1428] Setting key cae2e037f07c3f2e10ddb66ecc037ed0
I20260808 13:10:00.652868 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:10:00 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:00.664929 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Bootstrap starting.
I20260808 13:10:00.668246 783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:00.669507 783 log.cc:824] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:00.672441 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:00.652238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.31.59.190:46777)
0808 13:10:00.653012 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:00.653027 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:00.653075 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:10:00.655390 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:00.655418 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:00.655917 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:00.656359 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:00.657869 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:00.659358 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:00.660559 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:00.661202 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:00.661402 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:00.665423 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:00.665430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:00.665474 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:00.665516 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:00.668353 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:00.669159 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:00.669165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:00.669171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:00.669338 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:00.669741 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:00.669754 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:00.669756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:00.670427 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:00.670767 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:00.671057 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 13:10:00.671413 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":306,"threads_started":1}
I20260808 13:10:00.672652 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: No bootstrap required, opened a new log
I20260808 13:10:00.680462 783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } }
I20260808 13:10:00.680824 783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:00.680920 783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d50715ab5cf4363ba3b8cc466daa628, State: Initialized, Role: FOLLOWER
I20260808 13:10:00.681524 783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } }
I20260808 13:10:00.681759 783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:00.681885 783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:00.682046 783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:00.684142 783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } }
I20260808 13:10:00.684607 783 leader_election.cc:302] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d50715ab5cf4363ba3b8cc466daa628; no voters:
I20260808 13:10:00.685159 783 leader_election.cc:288] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:00.685281 789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:00.686515 789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [term 1 LEADER]: Becoming Leader. State: Replica: 1d50715ab5cf4363ba3b8cc466daa628, State: Running, Role: LEADER
I20260808 13:10:00.687054 789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } }
I20260808 13:10:00.687350 783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:00.689635 792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d50715ab5cf4363ba3b8cc466daa628. Latest consensus state: current_term: 1 leader_uuid: "1d50715ab5cf4363ba3b8cc466daa628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } } }
I20260808 13:10:00.690043 792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:00.689939 791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d50715ab5cf4363ba3b8cc466daa628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d50715ab5cf4363ba3b8cc466daa628" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 46777 } } }
I20260808 13:10:00.690191 791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:00.691704 799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:00.696760 799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:00.704950 799 catalog_manager.cc:1347] Generated new cluster ID: 2d9d25031fb94017bade6d4e16e88b55
I20260808 13:10:00.705068 799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:00.718119 799 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:00.719468 799 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:00.740084 799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Generated new TSK 0
I20260808 13:10:00.740953 799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:00.792681 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:46777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:01.112313 813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:01.112592 813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:01.112739 813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:01.120083 813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:01.120210 813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:01.120345 813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:01.120457 813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:01.130209 813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:46777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:01.131527 813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:01.132855 813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:01.145323 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:01.145406 821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:01.145341 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:01.146409 813 server_base.cc:1034] running on GCE node
I20260808 13:10:01.146983 813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:01.147832 813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:01.149036 813 hybrid_clock.cc:648] HybridClock initialized: now 1786194601148976 us; error 32 us; skew 500 ppm
I20260808 13:10:01.149299 813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:01 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194601, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:01.154912 813 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:01.156337 813 webserver.cc:460] Webserver started at http://127.31.59.129:34819/ using document root <none> and password file <none>
I20260808 13:10:01.156983 813 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:01.157081 813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:01.157387 813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:01.160189 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "728996d21cbd412c8be23b0837dec4d8"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "79836c79405fb72ffc53f27c3cc8573f"
server_key_iv: "436702ce5a1c928d28add9f14f1126ab"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.161033 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "728996d21cbd412c8be23b0837dec4d8"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "79836c79405fb72ffc53f27c3cc8573f"
server_key_iv: "436702ce5a1c928d28add9f14f1126ab"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.166513 813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:10:01.170393 828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:01.173115 813 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 13:10:01.173300 813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "728996d21cbd412c8be23b0837dec4d8"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "79836c79405fb72ffc53f27c3cc8573f"
server_key_iv: "436702ce5a1c928d28add9f14f1126ab"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.173581 813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:01.202229 813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:01.208323 813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:01.208665 813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:01.212244 813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:01.212349 813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:01.212471 813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:01.212514 813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:01 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194601, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:01.239988 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.212290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:58117 (local address 127.31.59.190:46777)
0808 13:10:01.222103 (+ 9813us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:01.222111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:01.222133 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:10:01.222207 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:01.222213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:01.222306 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:01.222420 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:01.224047 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.225144 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.227525 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.228188 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.228322 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.232951 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:01.232957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:01.232963 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:01.233007 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:01.236098 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.237065 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.237073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.237077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.237257 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.237800 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.237805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.237808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.238151 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:01.238302 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:01.239327 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0808 13:10:01.239550 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9673,"thread_start_us":9540,"threads_started":1}
I20260808 13:10:01.241086 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.213028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.129:58117)
0808 13:10:01.218587 (+ 5559us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:01.218643 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:01.221319 (+ 2676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:01.222673 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:01.222682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:01.223246 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:01.223856 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.223877 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.225290 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.225294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.227304 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.227316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.228457 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.228465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.228765 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.229555 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:01.229582 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:01.232663 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:01.236251 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.236261 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.236283 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.236908 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.237362 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.237368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.237372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.237674 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.238355 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:01.238364 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:01.238884 (+ 520us) client_negotiation.cc:769] Sending connection context
0808 13:10:01.239221 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 13:10:01.239770 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5214,"thread_start_us":4969,"threads_started":1}
I20260808 13:10:01.260600 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.246937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54665 (local address 127.31.59.190:46777)
0808 13:10:01.247075 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:01.247079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:01.247705 (+ 626us) server_negotiation.cc:407] Connection header received
0808 13:10:01.247885 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:01.247891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:01.247957 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:01.248035 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:01.248929 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.249707 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.251449 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.252102 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.252235 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.254318 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:01.254325 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:01.254331 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:01.254372 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:01.257624 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.258410 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.258416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.258421 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.258614 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.259030 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.259036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.259039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.259369 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:01.259524 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:01.260169 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 13:10:01.260355 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 13:10:01.260977 905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.247027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.129:54665)
0808 13:10:01.247609 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:01.247625 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:01.247774 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:01.248071 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:01.248075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:01.248513 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:01.248799 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.248808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.249841 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.249845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.251293 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.251308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.252306 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.252317 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.252527 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.253176 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:01.253208 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:01.254138 (+ 930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:01.257771 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.257777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.257781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.258279 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.258732 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.258740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.258743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.258923 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.259507 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:01.259514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:01.259814 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 13:10:01.260058 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 13:10:01.260455 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":338,"threads_started":1}
I20260808 13:10:01.270408 813 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:40329
I20260808 13:10:01.270429 959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:40329 every 8 connection(s)
I20260808 13:10:01.271639 813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:01.273701 961 heartbeater.cc:347] Connected to a master server at 127.31.59.190:46777
I20260808 13:10:01.273926 961 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:01.274387 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 813
I20260808 13:10:01.274622 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:01.274725 961 heartbeater.cc:511] Master 127.31.59.190:46777 requested a full tablet report, sending...
I20260808 13:10:01.275449 31982 external_mini_cluster.cc:1428] Setting key 53a946536a759d05d679d85616e27d15
I20260808 13:10:01.276957 735 ts_manager.cc:194] Registered new tserver with Master: 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329)
I20260808 13:10:01.278510 735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:58117
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:01.316120 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:46777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:01.714360 965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:01.714654 965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:01.714793 965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:01.721983 965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:01.722106 965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:01.722237 965 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:01.722347 965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:01.732190 965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:46777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:01.733494 965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:01.734869 965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:01.747308 970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:01.747359 971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:01.747386 973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:01.748345 965 server_base.cc:1034] running on GCE node
I20260808 13:10:01.748907 965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:01.749791 965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:01.751036 965 hybrid_clock.cc:648] HybridClock initialized: now 1786194601750970 us; error 36 us; skew 500 ppm
I20260808 13:10:01.751335 965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:01 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194601, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:01.759624 965 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:01.761166 965 webserver.cc:460] Webserver started at http://127.31.59.130:34967/ using document root <none> and password file <none>
I20260808 13:10:01.761936 965 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:01.762036 965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:01.762431 965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:01.765336 965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "f1cbe5adab9d41889c55c5edecb2360f"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "332911c18c1f2f6d3b22f0e47d3f3d07"
server_key_iv: "990327b023eb0e2e8be573af46e5db17"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.766346 965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "f1cbe5adab9d41889c55c5edecb2360f"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "332911c18c1f2f6d3b22f0e47d3f3d07"
server_key_iv: "990327b023eb0e2e8be573af46e5db17"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.771310 965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:01.774778 980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:01.776183 965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:01.776347 965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "f1cbe5adab9d41889c55c5edecb2360f"
format_stamp: "Formatted at 2026-08-08 13:10:01 on dist-test-slave-49h3"
server_key: "332911c18c1f2f6d3b22f0e47d3f3d07"
server_key_iv: "990327b023eb0e2e8be573af46e5db17"
server_key_version: "encryptionkey@0"
I20260808 13:10:01.776544 965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:01.794626 965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:01.798605 965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:01.798856 965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:01.801470 965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:01.801573 965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:01.801716 965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:01.801787 965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:01 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194601, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:01.824168 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.801894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:47911 (local address 127.31.59.190:46777)
0808 13:10:01.802663 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:01.802669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:01.803545 (+ 876us) server_negotiation.cc:407] Connection header received
0808 13:10:01.806387 (+ 2842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:01.806393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:01.806510 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:01.806622 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:01.808899 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.809972 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.812799 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.813305 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.813403 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.817721 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:01.817736 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:01.817741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:01.817785 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:01.820603 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.821263 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.821268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.821272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.821396 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.821828 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.821834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.821837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.822189 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:01.822323 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:01.823458 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0808 13:10:01.823710 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":516,"threads_started":1}
I20260808 13:10:01.824687 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.802506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.130:47911)
0808 13:10:01.803359 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:01.803418 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:01.806058 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:01.807096 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:01.807108 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:01.807807 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:01.808716 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.808747 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.810189 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.810194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.812631 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.812652 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.813557 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.813565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.813858 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.814542 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:01.814571 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:01.817513 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:01.820733 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.820741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.820756 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.821149 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.821510 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.821515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.821519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.821720 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.822323 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:01.822330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:01.822688 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 13:10:01.823039 (+ 351us) client_negotiation.cc:240] Negotiation successful
0808 13:10:01.823486 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":343,"threads_started":1}
I20260808 13:10:01.844339 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.830916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54435 (local address 127.31.59.190:46777)
0808 13:10:01.831112 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:01.831119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:01.831715 (+ 596us) server_negotiation.cc:407] Connection header received
0808 13:10:01.831878 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:01.831883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:01.831943 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:01.832019 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:01.832891 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.833715 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.835652 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.836090 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.836196 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.838130 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:01.838136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:01.838140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:01.838170 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:01.841218 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.842183 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.842188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.842192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.842327 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:01.842680 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:01.842685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:01.842689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:01.843063 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:01.843204 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:01.843783 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 13:10:01.844010 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:10:01.844411 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:01.831001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.130:54435)
0808 13:10:01.831625 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:01.831644 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:01.831774 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:01.832053 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:01.832057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:01.832487 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:01.832762 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.832771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.833912 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.833918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.835519 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:01.835534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:01.836264 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:01.836270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:01.836423 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:01.837030 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:01.837045 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:01.837891 (+ 846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:01.841378 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.841383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.841387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.842021 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.842456 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:01.842459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:01.842462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:01.842576 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:01.843183 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:01.843187 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:01.843432 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 13:10:01.843577 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 13:10:01.843950 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":337,"threads_started":1}
I20260808 13:10:01.852543 965 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:33229
I20260808 13:10:01.852566 1108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:33229 every 8 connection(s)
I20260808 13:10:01.853794 965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:01.854110 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 965
I20260808 13:10:01.854312 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:01.854887 31982 external_mini_cluster.cc:1428] Setting key 19033beba63505471108dace5715172d
I20260808 13:10:01.856022 1109 heartbeater.cc:347] Connected to a master server at 127.31.59.190:46777
I20260808 13:10:01.856242 1109 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:01.856953 1109 heartbeater.cc:511] Master 127.31.59.190:46777 requested a full tablet report, sending...
I20260808 13:10:01.858389 735 ts_manager.cc:194] Registered new tserver with Master: f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:01.859480 735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:47911
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:01.898347 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:46777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:02.216133 1113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:02.216426 1113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:02.216553 1113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:02.223654 1113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:02.223784 1113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:02.223908 1113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:02.224020 1113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:02.233635 1113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:46777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:02.234941 1113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:02.236285 1113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:02.248801 1118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:02.248831 1119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:02.248762 1121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:02.249804 1113 server_base.cc:1034] running on GCE node
I20260808 13:10:02.250422 1113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:02.251302 1113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:02.252518 1113 hybrid_clock.cc:648] HybridClock initialized: now 1786194602252454 us; error 34 us; skew 500 ppm
I20260808 13:10:02.252794 1113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:02 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:02.255749 1113 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:02.257025 1113 webserver.cc:460] Webserver started at http://127.31.59.131:38635/ using document root <none> and password file <none>
I20260808 13:10:02.257722 1113 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:02.257805 1113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:02.258067 1113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:02.260716 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "9922b1cbd55a4dbfba9027a9ffaebc72"
format_stamp: "Formatted at 2026-08-08 13:10:02 on dist-test-slave-49h3"
server_key: "efda8d57b9698d96df2ca13de1ddded2"
server_key_iv: "6c988c34c1cc67135508e49198b717ee"
server_key_version: "encryptionkey@0"
I20260808 13:10:02.261472 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "9922b1cbd55a4dbfba9027a9ffaebc72"
format_stamp: "Formatted at 2026-08-08 13:10:02 on dist-test-slave-49h3"
server_key: "efda8d57b9698d96df2ca13de1ddded2"
server_key_iv: "6c988c34c1cc67135508e49198b717ee"
server_key_version: "encryptionkey@0"
I20260808 13:10:02.266204 1113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:10:02.269352 1128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:02.270740 1113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:10:02.270901 1113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "9922b1cbd55a4dbfba9027a9ffaebc72"
format_stamp: "Formatted at 2026-08-08 13:10:02 on dist-test-slave-49h3"
server_key: "efda8d57b9698d96df2ca13de1ddded2"
server_key_iv: "6c988c34c1cc67135508e49198b717ee"
server_key_version: "encryptionkey@0"
I20260808 13:10:02.271091 1113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:02.281665 961 heartbeater.cc:503] Master 127.31.59.190:46777 was elected leader, sending a full tablet report...
I20260808 13:10:02.287060 1113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:02.291239 1113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:02.291458 1113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:02.293982 1113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:02.294060 1113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:02.294170 1113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:02.294221 1113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:02 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:02.315387 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.295602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:42385 (local address 127.31.59.190:46777)
0808 13:10:02.295786 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.295792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.296175 (+ 383us) server_negotiation.cc:407] Connection header received
0808 13:10:02.298320 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.298327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.298428 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.298506 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:02.300760 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.301784 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.304658 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.305346 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.305494 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.308856 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:02.308861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:02.308864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:02.308896 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:02.311851 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.312856 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.312861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.312865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.313027 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.313590 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.313595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.313598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.313950 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:02.314088 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.314793 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.315015 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:10:02.316524 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.295013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.131:42385)
0808 13:10:02.296006 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.296056 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.298006 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.299042 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.299056 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.299745 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:02.300567 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.300598 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.301963 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.301969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.304485 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.304506 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.305655 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.305666 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.305905 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.306562 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:02.306583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:02.308650 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:02.311985 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:02.311993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:02.312012 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:02.312726 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:02.313165 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:02.313170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:02.313173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:02.313468 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:02.314079 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:02.314086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:02.314419 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.314653 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.315154 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":329,"threads_started":1}
I20260808 13:10:02.336452 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.322767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:40027 (local address 127.31.59.190:46777)
0808 13:10:02.322971 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.322977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.323863 (+ 886us) server_negotiation.cc:407] Connection header received
0808 13:10:02.323978 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.323984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.324062 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.324167 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:02.324976 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.326145 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.327738 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.328415 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.328547 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.330833 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:02.330840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:02.330845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:02.330878 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:02.333667 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.334480 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.334483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.334486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.334625 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.335110 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.335115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.335119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.335436 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:02.335585 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.336016 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.336206 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:10:02.336815 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.323080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.31.59.131:40027)
0808 13:10:02.323776 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.323791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.323896 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.324174 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.324177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.324568 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:02.324860 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.324869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.326343 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.326349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.327608 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.327620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.328615 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.328621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.328821 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.329593 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:02.329619 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:02.330649 (+ 1030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:02.333797 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:02.333803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:02.333808 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:02.334373 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:02.334756 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:02.334761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:02.334764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:02.334935 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:02.335572 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:02.335578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:02.335786 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.335949 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.336317 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":372,"threads_started":1}
I20260808 13:10:02.343060 1113 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:35893
I20260808 13:10:02.343127 1254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:35893 every 8 connection(s)
I20260808 13:10:02.344187 1113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:02.346045 1255 heartbeater.cc:347] Connected to a master server at 127.31.59.190:46777
I20260808 13:10:02.346256 1255 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:02.346848 1255 heartbeater.cc:511] Master 127.31.59.190:46777 requested a full tablet report, sending...
I20260808 13:10:02.347438 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 1113
I20260808 13:10:02.347617 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:02.348117 31982 external_mini_cluster.cc:1428] Setting key c5f0a77d9343a7bcf5068b17cbf7f4f8
I20260808 13:10:02.348202 735 ts_manager.cc:194] Registered new tserver with Master: 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131:35893)
I20260808 13:10:02.349197 735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:42385
I20260808 13:10:02.351061 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:10:02 dist-test-slave-49h3 krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:10:02 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:02.397568 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.385125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.31.59.190:46777)
0808 13:10:02.385368 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.385374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.385594 (+ 220us) server_negotiation.cc:407] Connection header received
0808 13:10:02.385755 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.385758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.385824 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.385909 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:02.387186 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.387955 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.389195 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.389626 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.389718 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.393173 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:02.393178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:02.393181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:02.393215 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:02.395482 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.396003 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.396007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.396009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.396124 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:02.396405 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:02.396408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:02.396410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:02.396655 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:02.396857 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.397027 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.397194 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 13:10:02.401348 735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:02.408318 735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:02.447190 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.420013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33728 (local address 127.31.59.129:40329)
0808 13:10:02.439034 (+ 19021us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.439042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.439068 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:10:02.439148 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.439154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.439343 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.439519 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:02.440265 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.441811 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.444657 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.446017 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.446170 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.446389 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.446737 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.446893 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18884,"thread_start_us":18713,"threads_started":1}
I20260808 13:10:02.447346 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.419774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40329 (local address 127.0.0.1:33728)
0808 13:10:02.435873 (+ 16099us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.435925 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.436151 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.439561 (+ 3410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.439568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.439599 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:02.440043 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.440058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.441992 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.441998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.444484 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.444502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.446276 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.446283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.446504 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.446555 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.446663 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.446872 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15843,"mutex_wait_us":4,"thread_start_us":15686,"threads_started":1}
I20260808 13:10:02.448194 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.420664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.31.59.130:33229)
0808 13:10:02.437862 (+ 17198us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.437873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.440387 (+ 2514us) server_negotiation.cc:407] Connection header received
0808 13:10:02.440488 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.440495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.440744 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.440924 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:02.441575 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.442927 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.445013 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.446155 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.446297 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.446487 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.447768 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.447927 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17000,"thread_start_us":16866,"threads_started":1}
I20260808 13:10:02.447389 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.420190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:33229 (local address 127.0.0.1:39084)
0808 13:10:02.440141 (+ 19951us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.440159 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.440263 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.441095 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.441098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.441117 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:02.441408 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.441416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.443090 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.443094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.444849 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.444865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.446550 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.446557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.446715 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.446758 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.446842 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.446974 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19825,"mutex_wait_us":1,"thread_start_us":17866,"threads_started":1}
I20260808 13:10:02.449978 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.419943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:35893 (local address 127.0.0.1:58582)
0808 13:10:02.438378 (+ 18435us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.438403 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.438530 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.441938 (+ 3408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.441944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.441969 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:02.442451 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.442465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.445333 (+ 2868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.445339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.447378 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.447393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.449312 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.449318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.449516 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.449549 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.449637 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.449733 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18289,"mutex_wait_us":22,"thread_start_us":19906,"threads_started":1}
I20260808 13:10:02.450601 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.420444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.31.59.131:35893)
0808 13:10:02.441395 (+ 20951us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.441403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.441433 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:10:02.441540 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.441546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.441727 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.441931 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:02.442661 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.445202 (+ 2541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.447564 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.448828 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.448950 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.449152 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.450169 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.450350 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20779,"thread_start_us":21115,"threads_started":1}
I20260808 13:10:02.451435 884 tablet_service.cc:1467] Processing CreateTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb (DEFAULT_TABLE table=test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.451857 1034 tablet_service.cc:1467] Processing CreateTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb (DEFAULT_TABLE table=test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.453002 884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ca9388f56b4282846ad18b2a4d9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.453127 1034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ca9388f56b4282846ad18b2a4d9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.453388 1181 tablet_service.cc:1467] Processing CreateTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb (DEFAULT_TABLE table=test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.454638 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ca9388f56b4282846ad18b2a4d9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.465600 1274 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Bootstrap starting.
I20260808 13:10:02.468768 1274 tablet_bootstrap.cc:654] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.469630 1276 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Bootstrap starting.
I20260808 13:10:02.470075 1274 log.cc:824] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:02.470487 1277 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Bootstrap starting.
I20260808 13:10:02.473086 1274 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: No bootstrap required, opened a new log
I20260808 13:10:02.473328 1274 ts_tablet_manager.cc:1397] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:10:02.474119 1277 tablet_bootstrap.cc:654] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.474328 1276 tablet_bootstrap.cc:654] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.475400 1277 log.cc:824] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:02.475521 1276 log.cc:824] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:02.478029 1277 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: No bootstrap required, opened a new log
I20260808 13:10:02.478288 1277 ts_tablet_manager.cc:1397] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:10:02.478372 1276 tablet_bootstrap.cc:492] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: No bootstrap required, opened a new log
I20260808 13:10:02.478621 1276 ts_tablet_manager.cc:1397] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 13:10:02.483126 1274 raft_consensus.cc:348] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.483554 1274 raft_consensus.cc:374] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.483650 1274 raft_consensus.cc:729] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.484251 1274 consensus_queue.cc:260] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.486267 1277 raft_consensus.cc:348] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.486569 1277 raft_consensus.cc:374] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.486663 1277 raft_consensus.cc:729] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cbe5adab9d41889c55c5edecb2360f, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.487164 1277 consensus_queue.cc:260] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.488216 1109 heartbeater.cc:503] Master 127.31.59.190:46777 was elected leader, sending a full tablet report...
I20260808 13:10:02.488718 1277 ts_tablet_manager.cc:1428] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260808 13:10:02.489523 1255 heartbeater.cc:503] Master 127.31.59.190:46777 was elected leader, sending a full tablet report...
I20260808 13:10:02.490207 1274 ts_tablet_manager.cc:1428] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260808 13:10:02.490475 1276 raft_consensus.cc:348] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.490906 1276 raft_consensus.cc:374] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.491003 1276 raft_consensus.cc:729] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 728996d21cbd412c8be23b0837dec4d8, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.492344 1276 consensus_queue.cc:260] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.494509 1276 ts_tablet_manager.cc:1428] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
W20260808 13:10:02.524145 962 tablet.cc:2369] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:02.595698 1256 tablet.cc:2369] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:02.605896 1110 tablet.cc:2369] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:02.753571 1282 raft_consensus.cc:482] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:02.753953 1282 raft_consensus.cc:504] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.755856 1282 leader_election.cc:288] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:02.761548 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.756470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:44539 (local address 127.31.59.130:33229)
0808 13:10:02.756809 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.756813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.756829 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:10:02.756893 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.756897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.756954 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.757035 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:02.757495 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.758235 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.759836 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.760624 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.760782 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.760860 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.761122 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.761264 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260808 13:10:02.761597 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.756117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:33229 (local address 127.31.59.131:44539)
0808 13:10:02.756392 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.756413 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.756557 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.757089 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.757094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.757159 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:02.757326 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.757334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.758366 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.758371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.759668 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.759681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.760783 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.760789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.760972 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.760996 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.761079 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.761220 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":3,"spinlock_wait_cycles":52736}
I20260808 13:10:02.761904 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.756443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:48213 (local address 127.31.59.129:40329)
0808 13:10:02.756653 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.756659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.757101 (+ 442us) server_negotiation.cc:407] Connection header received
0808 13:10:02.757292 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.757296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.757369 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.757470 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:02.757986 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.758678 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.760261 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.761024 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.761171 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.761244 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.761531 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.761662 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:10:02.762024 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.756270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40329 (local address 127.31.59.131:48213)
0808 13:10:02.756989 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:02.757004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:02.757165 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:02.757506 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:02.757511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:02.757536 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:02.757829 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.757837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.758817 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.758821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.760113 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:02.760124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.761234 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.761239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:02.761376 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.761400 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:10:02.761500 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 13:10:02.761632 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"spinlock_wait_cycles":14848,"thread_start_us":422,"threads_started":1}
I20260808 13:10:02.762984 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ca9388f56b4282846ad18b2a4d9bdb" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f" is_pre_election: true
I20260808 13:10:02.762984 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ca9388f56b4282846ad18b2a4d9bdb" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8" is_pre_election: true
I20260808 13:10:02.763582 1061 raft_consensus.cc:2454] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:02.763582 911 raft_consensus.cc:2454] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:02.764313 1133 leader_election.cc:302] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 728996d21cbd412c8be23b0837dec4d8, 9922b1cbd55a4dbfba9027a9ffaebc72, f1cbe5adab9d41889c55c5edecb2360f; no voters:
I20260808 13:10:02.764705 1282 raft_consensus.cc:2781] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:02.764861 1282 raft_consensus.cc:482] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:02.764966 1282 raft_consensus.cc:3037] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:02.767131 1282 raft_consensus.cc:504] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.767717 1282 leader_election.cc:288] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Requested vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:02.768131 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ca9388f56b4282846ad18b2a4d9bdb" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8"
I20260808 13:10:02.768221 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ca9388f56b4282846ad18b2a4d9bdb" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f"
I20260808 13:10:02.768312 911 raft_consensus.cc:3037] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:02.768436 1061 raft_consensus.cc:3037] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:02.770642 911 raft_consensus.cc:2454] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:02.770781 1061 raft_consensus.cc:2454] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:02.771059 1132 leader_election.cc:302] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 728996d21cbd412c8be23b0837dec4d8, 9922b1cbd55a4dbfba9027a9ffaebc72; no voters:
I20260808 13:10:02.771394 1282 raft_consensus.cc:2781] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:02.771752 1282 raft_consensus.cc:686] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 LEADER]: Becoming Leader. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Running, Role: LEADER
I20260808 13:10:02.772198 1282 consensus_queue.cc:237] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } }
I20260808 13:10:02.776521 734 catalog_manager.cc:5495] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131). New cstate: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:02.824350 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:02.818957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.31.59.131:35893)
0808 13:10:02.819156 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:02.819162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:02.819241 (+ 79us) server_negotiation.cc:407] Connection header received
0808 13:10:02.819559 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:02.819563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:02.819652 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:02.819769 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:02.820141 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.820860 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.822239 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:02.822650 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:02.822757 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:02.823265 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:02.823745 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:02.823872 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:02.824006 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 13:10:02.824087 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:10:02.832623 911 raft_consensus.cc:1261] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:02.832623 1061 raft_consensus.cc:1261] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:02.833657 1286 consensus_queue.cc:1035] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:02.834023 1282 consensus_queue.cc:1035] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:02.868614 735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:02.868933 735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:02.873260 735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Sending DeleteTablet for 3 replicas of tablet a0ca9388f56b4282846ad18b2a4d9bdb
I20260808 13:10:02.874492 884 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:02 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33728
I20260808 13:10:02.875111 1034 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:02 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:39084
I20260808 13:10:02.887936 1181 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ca9388f56b4282846ad18b2a4d9bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:02 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:58582
I20260808 13:10:02.889878 1299 tablet_replica.cc:330] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: stopping tablet replica
I20260808 13:10:02.890426 1299 raft_consensus.cc:2229] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:02.890985 1299 raft_consensus.cc:2258] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:02.892082 1301 tablet_replica.cc:330] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: stopping tablet replica
I20260808 13:10:02.892554 1301 raft_consensus.cc:2229] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:02.893124 1301 raft_consensus.cc:2258] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:02.894295 735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:02.895105 735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:02.895566 1301 ts_tablet_manager.cc:1905] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:02.896392 1299 ts_tablet_manager.cc:1905] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:02.904556 1299 ts_tablet_manager.cc:1918] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:02.904878 1299 log.cc:1196] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/a0ca9388f56b4282846ad18b2a4d9bdb
I20260808 13:10:02.904893 1301 ts_tablet_manager.cc:1918] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:02.905324 1301 log.cc:1196] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/a0ca9388f56b4282846ad18b2a4d9bdb
I20260808 13:10:02.905550 1302 tablet_replica.cc:330] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: stopping tablet replica
I20260808 13:10:02.906008 1299 ts_tablet_manager.cc:1939] T a0ca9388f56b4282846ad18b2a4d9bdb P 728996d21cbd412c8be23b0837dec4d8: Deleting consensus metadata
I20260808 13:10:02.906172 1302 raft_consensus.cc:2229] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:02.906726 1301 ts_tablet_manager.cc:1939] T a0ca9388f56b4282846ad18b2a4d9bdb P f1cbe5adab9d41889c55c5edecb2360f: Deleting consensus metadata
I20260808 13:10:02.907284 1302 raft_consensus.cc:2258] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:02.908632 719 catalog_manager.cc:4841] TS 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329): tablet a0ca9388f56b4282846ad18b2a4d9bdb (table test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]) successfully deleted
I20260808 13:10:02.909242 720 catalog_manager.cc:4841] TS f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229): tablet a0ca9388f56b4282846ad18b2a4d9bdb (table test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]) successfully deleted
I20260808 13:10:02.909765 884 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 (DEFAULT_TABLE table=test-table [id=5b6e8623113b44a8aaae4d358afdf64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.910208 1034 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 (DEFAULT_TABLE table=test-table [id=5b6e8623113b44a8aaae4d358afdf64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.910444 884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ddbefacda4fe48f15fd3d448e7277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.910835 1034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ddbefacda4fe48f15fd3d448e7277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.912415 1181 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 (DEFAULT_TABLE table=test-table [id=5b6e8623113b44a8aaae4d358afdf64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:02.913617 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ddbefacda4fe48f15fd3d448e7277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:02.915186 1302 ts_tablet_manager.cc:1905] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:02.919492 1276 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Bootstrap starting.
I20260808 13:10:02.919545 1277 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Bootstrap starting.
I20260808 13:10:02.922365 1277 tablet_bootstrap.cc:654] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.922370 1276 tablet_bootstrap.cc:654] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.932152 1276 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: No bootstrap required, opened a new log
I20260808 13:10:02.932350 1276 ts_tablet_manager.cc:1397] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260808 13:10:02.933887 1276 raft_consensus.cc:348] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.934249 1276 raft_consensus.cc:374] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.934505 1276 raft_consensus.cc:729] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 728996d21cbd412c8be23b0837dec4d8, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.934912 1276 consensus_queue.cc:260] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.935380 1274 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Bootstrap starting.
I20260808 13:10:02.938584 1274 tablet_bootstrap.cc:654] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:02.938856 1276 ts_tablet_manager.cc:1428] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260808 13:10:02.943909 1277 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: No bootstrap required, opened a new log
I20260808 13:10:02.944211 1277 ts_tablet_manager.cc:1397] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.016s
I20260808 13:10:02.945734 1302 ts_tablet_manager.cc:1918] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:02.945760 1277 raft_consensus.cc:348] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.946028 1302 log.cc:1196] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/a0ca9388f56b4282846ad18b2a4d9bdb
I20260808 13:10:02.946084 1277 raft_consensus.cc:374] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.946218 1277 raft_consensus.cc:729] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cbe5adab9d41889c55c5edecb2360f, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.946728 1277 consensus_queue.cc:260] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.947021 1302 ts_tablet_manager.cc:1939] T a0ca9388f56b4282846ad18b2a4d9bdb P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting consensus metadata
I20260808 13:10:02.949014 1277 ts_tablet_manager.cc:1428] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260808 13:10:02.949633 717 catalog_manager.cc:4841] TS 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131:35893): tablet a0ca9388f56b4282846ad18b2a4d9bdb (table test-table [id=74ed0dbbf3414631a2c8a0e8c9ecf997]) successfully deleted
I20260808 13:10:02.951575 1274 tablet_bootstrap.cc:492] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: No bootstrap required, opened a new log
I20260808 13:10:02.951829 1274 ts_tablet_manager.cc:1397] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.006s
I20260808 13:10:02.952986 1274 raft_consensus.cc:348] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.953261 1274 raft_consensus.cc:374] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:02.953385 1274 raft_consensus.cc:729] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Initialized, Role: FOLLOWER
I20260808 13:10:02.953768 1274 consensus_queue.cc:260] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:02.954458 1274 ts_tablet_manager.cc:1428] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 13:10:03.139454 1286 raft_consensus.cc:482] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:03.139806 1286 raft_consensus.cc:504] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.140683 1286 leader_election.cc:288] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:03.141438 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ddbefacda4fe48f15fd3d448e7277" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f" is_pre_election: true
I20260808 13:10:03.141422 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ddbefacda4fe48f15fd3d448e7277" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8" is_pre_election: true
I20260808 13:10:03.141737 911 raft_consensus.cc:2454] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:03.141880 1061 raft_consensus.cc:2454] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:03.142247 1132 leader_election.cc:302] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 728996d21cbd412c8be23b0837dec4d8, 9922b1cbd55a4dbfba9027a9ffaebc72; no voters:
I20260808 13:10:03.142712 1286 raft_consensus.cc:2781] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:03.142968 1286 raft_consensus.cc:482] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:03.143445 1286 raft_consensus.cc:3037] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.148626 1286 raft_consensus.cc:504] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.149785 1286 leader_election.cc:288] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Requested vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:03.150247 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ddbefacda4fe48f15fd3d448e7277" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8"
I20260808 13:10:03.150455 911 raft_consensus.cc:3037] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.150451 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ddbefacda4fe48f15fd3d448e7277" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f"
I20260808 13:10:03.150856 1061 raft_consensus.cc:3037] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.152519 911 raft_consensus.cc:2454] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:03.153111 1132 leader_election.cc:302] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 728996d21cbd412c8be23b0837dec4d8, 9922b1cbd55a4dbfba9027a9ffaebc72; no voters:
I20260808 13:10:03.153388 1286 raft_consensus.cc:2781] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:03.153604 1286 raft_consensus.cc:686] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 LEADER]: Becoming Leader. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Running, Role: LEADER
I20260808 13:10:03.153952 1286 consensus_queue.cc:237] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.155114 1061 raft_consensus.cc:2454] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:03.157662 735 catalog_manager.cc:5495] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131). New cstate: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } health_report { overall_health: HEALTHY } } }
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194600, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:03.270900 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.246971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:46777 (local address 127.0.0.1:47742)
0808 13:10:03.248305 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:03.248328 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:03.248560 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:03.249008 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:03.249013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:03.250460 (+ 1447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:03.251075 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.251095 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.252836 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.252842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.255169 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.255185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.256244 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.256252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.256615 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.257527 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:03.257555 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:03.261762 (+ 4207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:03.266702 (+ 4940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:03.266745 (+ 43us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:03.266754 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:03.267669 (+ 915us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:03.268135 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:03.268140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:03.268143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:03.268485 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:03.269153 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:03.269162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:03.269543 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 13:10:03.269827 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 13:10:03.270182 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":878,"threads_started":1}
I20260808 13:10:03.271030 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.245798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.31.59.190:46777)
0808 13:10:03.247163 (+ 1365us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.247170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.248435 (+ 1265us) server_negotiation.cc:407] Connection header received
0808 13:10:03.248698 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.248703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.248782 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.248893 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.251307 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.252631 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.255333 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.256025 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.256143 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.262004 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.262011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.262015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.262054 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.266523 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.267799 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.267802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.267805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.268016 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.268602 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.268605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.268608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.269006 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.269116 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.269936 (+ 820us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.270332 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":1101,"thread_start_us":794,"threads_started":1}
I20260808 13:10:03.278613 735 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:47742:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:10:03.280143 735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:03.289659 1181 tablet_service.cc:1467] Processing CreateTablet for tablet ee68501be8b14919a81f2ad2b7ecf603 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc78eb8091bb4ecea2e1d4395e65ec53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:03.289673 884 tablet_service.cc:1467] Processing CreateTablet for tablet ee68501be8b14919a81f2ad2b7ecf603 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc78eb8091bb4ecea2e1d4395e65ec53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:03.290372 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee68501be8b14919a81f2ad2b7ecf603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:03.290449 884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee68501be8b14919a81f2ad2b7ecf603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:03.297883 1034 tablet_service.cc:1467] Processing CreateTablet for tablet ee68501be8b14919a81f2ad2b7ecf603 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc78eb8091bb4ecea2e1d4395e65ec53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:03.298756 1034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee68501be8b14919a81f2ad2b7ecf603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:03.300397 1276 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: Bootstrap starting.
I20260808 13:10:03.303130 1276 tablet_bootstrap.cc:654] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:03.305563 1274 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: Bootstrap starting.
I20260808 13:10:03.307163 1276 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: No bootstrap required, opened a new log
I20260808 13:10:03.307453 1276 ts_tablet_manager.cc:1397] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260808 13:10:03.308576 1277 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: Bootstrap starting.
I20260808 13:10:03.308717 1276 raft_consensus.cc:348] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.309067 1274 tablet_bootstrap.cc:654] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:03.309089 1276 raft_consensus.cc:374] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:03.309243 1276 raft_consensus.cc:729] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 728996d21cbd412c8be23b0837dec4d8, State: Initialized, Role: FOLLOWER
I20260808 13:10:03.309711 1276 consensus_queue.cc:260] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.311710 1277 tablet_bootstrap.cc:654] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:03.312772 1284 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.314531 1284 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.314870 1276 ts_tablet_manager.cc:1428] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:03.323113 1274 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: No bootstrap required, opened a new log
I20260808 13:10:03.323341 1274 ts_tablet_manager.cc:1397] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.009s
I20260808 13:10:03.323388 1277 tablet_bootstrap.cc:492] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: No bootstrap required, opened a new log
I20260808 13:10:03.323685 1277 ts_tablet_manager.cc:1397] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260808 13:10:03.324548 1274 raft_consensus.cc:348] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.324901 1274 raft_consensus.cc:374] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:03.325155 1274 raft_consensus.cc:729] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Initialized, Role: FOLLOWER
I20260808 13:10:03.325068 1277 raft_consensus.cc:348] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.325467 1277 raft_consensus.cc:374] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:03.325654 1277 raft_consensus.cc:729] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cbe5adab9d41889c55c5edecb2360f, State: Initialized, Role: FOLLOWER
I20260808 13:10:03.325716 1274 consensus_queue.cc:260] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.326109 1277 consensus_queue.cc:260] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.326867 1277 ts_tablet_manager.cc:1428] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:03.326820 1283 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.327497 1283 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.328364 1286 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.328784 1286 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.328827 1274 ts_tablet_manager.cc:1428] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:03.354084 1286 raft_consensus.cc:482] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:03.354386 1286 raft_consensus.cc:504] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.355278 1286 leader_election.cc:288] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:03.355854 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee68501be8b14919a81f2ad2b7ecf603" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8" is_pre_election: true
I20260808 13:10:03.356201 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee68501be8b14919a81f2ad2b7ecf603" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f" is_pre_election: true
I20260808 13:10:03.356297 911 raft_consensus.cc:2454] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:03.356510 1061 raft_consensus.cc:2454] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 0.
I20260808 13:10:03.356854 1132 leader_election.cc:302] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 728996d21cbd412c8be23b0837dec4d8, 9922b1cbd55a4dbfba9027a9ffaebc72; no voters:
I20260808 13:10:03.357257 1286 raft_consensus.cc:2781] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:03.357759 1286 raft_consensus.cc:482] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:03.358003 1286 raft_consensus.cc:3037] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.361344 1286 raft_consensus.cc:504] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.362567 1061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee68501be8b14919a81f2ad2b7ecf603" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cbe5adab9d41889c55c5edecb2360f"
I20260808 13:10:03.362581 911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee68501be8b14919a81f2ad2b7ecf603" candidate_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "728996d21cbd412c8be23b0837dec4d8"
I20260808 13:10:03.362939 1061 raft_consensus.cc:3037] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.362974 911 raft_consensus.cc:3037] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:03.366662 911 raft_consensus.cc:2454] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:03.366662 1061 raft_consensus.cc:2454] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9922b1cbd55a4dbfba9027a9ffaebc72 in term 1.
I20260808 13:10:03.367237 1133 leader_election.cc:302] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9922b1cbd55a4dbfba9027a9ffaebc72, f1cbe5adab9d41889c55c5edecb2360f; no voters:
I20260808 13:10:03.367484 1286 leader_election.cc:288] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [CANDIDATE]: Term 1 election: Requested vote from peers 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329), f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229)
I20260808 13:10:03.367806 1282 raft_consensus.cc:2781] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:03.368050 1282 raft_consensus.cc:686] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 LEADER]: Becoming Leader. State: Replica: 9922b1cbd55a4dbfba9027a9ffaebc72, State: Running, Role: LEADER
I20260808 13:10:03.368482 1282 consensus_queue.cc:237] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } }
I20260808 13:10:03.371459 1286 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: TxnStatusTablet state changed. Reason: New leader 9922b1cbd55a4dbfba9027a9ffaebc72. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.372064 1286 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:03.373401 735 catalog_manager.cc:5495] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131). New cstate: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } health_report { overall_health: HEALTHY } } }
I20260808 13:10:03.377751 1318 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:10:03.378139 1318 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:10:03.407778 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.400019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:35893 (local address 127.0.0.1:58588)
0808 13:10:03.400567 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:03.400590 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:03.400755 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:03.401332 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:03.401337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:03.401350 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:03.401691 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.401702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.403466 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.403471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.405026 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.405041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.406219 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.406226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.406474 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.406534 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:03.407020 (+ 486us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:03.407046 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 13:10:03.407119 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 13:10:03.407295 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333}
I20260808 13:10:03.407950 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.399848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.31.59.131:35893)
0808 13:10:03.401034 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.401038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.401058 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:03.401114 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.401118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.401181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.401279 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:03.401858 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.403330 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.405181 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.406028 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.406187 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.406774 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:03.406930 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:03.407018 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.407155 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.407234 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":666,"threads_started":1}
I20260808 13:10:03.413822 1061 raft_consensus.cc:1261] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:03.414265 911 raft_consensus.cc:1261] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:03.414472 1286 consensus_queue.cc:1035] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:03.415016 1282 consensus_queue.cc:1035] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:03.424851 1283 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: TxnStatusTablet state changed. Reason: New leader 9922b1cbd55a4dbfba9027a9ffaebc72. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.425192 1283 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.426846 1284 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: TxnStatusTablet state changed. Reason: New leader 9922b1cbd55a4dbfba9027a9ffaebc72. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.427186 1284 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.431766 1283 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.432099 1283 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P f1cbe5adab9d41889c55c5edecb2360f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.432934 1284 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.433432 1284 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 728996d21cbd412c8be23b0837dec4d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:03.437295 1282 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.437927 1282 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:03.438133 1322 mvcc.cc:205] Tried to move back new op lower bound from 7316253095576268800 to 7316253095400456192. Current Snapshot: MvccSnapshot[applied={T|T < 7316253095576268800}]
I20260808 13:10:03.447892 1282 tablet_replica.cc:439] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 } } peers { permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 } } peers { permanent_uuid: "9922b1cbd55a4dbfba9027a9ffaebc72" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35893 } } }
I20260808 13:10:03.448515 1282 tablet_replica.cc:442] T ee68501be8b14919a81f2ad2b7ecf603 P 9922b1cbd55a4dbfba9027a9ffaebc72: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:03.482076 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.471176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:38705 (local address 127.31.59.131:35893)
0808 13:10:03.471661 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.471667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.473337 (+ 1670us) server_negotiation.cc:407] Connection header received
0808 13:10:03.473640 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.473645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.473734 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.473842 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:03.475318 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.476969 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.479613 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.480472 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.480589 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.481208 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:03.481319 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:03.481402 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.481576 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.481647 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260808 13:10:03.482558 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.471405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:35893 (local address 127.31.59.131:38705)
0808 13:10:03.473216 (+ 1811us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:03.473234 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:03.473397 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:03.473855 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:03.473858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:03.473868 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:03.474926 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.474943 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.477178 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.477182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.479456 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:03.479469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.480728 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.480735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:03.480946 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.481007 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:03.481434 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:03.481440 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:10:03.481558 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 13:10:03.481697 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"thread_start_us":1260,"threads_started":1}
I20260808 13:10:03.495702 911 raft_consensus.cc:1261] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:03.495754 1061 raft_consensus.cc:1261] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Refusing update from remote peer 9922b1cbd55a4dbfba9027a9ffaebc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:03.496533 1282 consensus_queue.cc:1035] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "728996d21cbd412c8be23b0837dec4d8" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:03.496879 1329 consensus_queue.cc:1035] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cbe5adab9d41889c55c5edecb2360f" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:03.574406 734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:03.574934 734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:03.578685 734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d50715ab5cf4363ba3b8cc466daa628: Sending DeleteTablet for 3 replicas of tablet 6a2ddbefacda4fe48f15fd3d448e7277
I20260808 13:10:03.579536 884 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33728
I20260808 13:10:03.579746 1034 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:39084
I20260808 13:10:03.580137 1181 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ddbefacda4fe48f15fd3d448e7277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:58582
I20260808 13:10:03.590334 1341 tablet_replica.cc:330] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: stopping tablet replica
I20260808 13:10:03.592031 1341 raft_consensus.cc:2229] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:03.592993 1341 raft_consensus.cc:2258] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:03.593384 1342 tablet_replica.cc:330] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: stopping tablet replica
I20260808 13:10:03.593734 1342 raft_consensus.cc:2229] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:03.594497 1342 raft_consensus.cc:2258] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:03.595737 1341 ts_tablet_manager.cc:1905] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:03.595906 1342 ts_tablet_manager.cc:1905] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:03.600657 1340 tablet_replica.cc:330] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: stopping tablet replica
I20260808 13:10:03.600934 1340 raft_consensus.cc:2229] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:03.601440 1340 raft_consensus.cc:2258] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:03.606024 1342 ts_tablet_manager.cc:1918] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:03.606273 1342 log.cc:1196] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/6a2ddbefacda4fe48f15fd3d448e7277
I20260808 13:10:03.607065 1342 ts_tablet_manager.cc:1939] T 6a2ddbefacda4fe48f15fd3d448e7277 P 9922b1cbd55a4dbfba9027a9ffaebc72: Deleting consensus metadata
I20260808 13:10:03.607761 1341 ts_tablet_manager.cc:1918] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:03.607990 1341 log.cc:1196] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/6a2ddbefacda4fe48f15fd3d448e7277
I20260808 13:10:03.608385 717 catalog_manager.cc:4841] TS 9922b1cbd55a4dbfba9027a9ffaebc72 (127.31.59.131:35893): tablet 6a2ddbefacda4fe48f15fd3d448e7277 (table test-table [id=5b6e8623113b44a8aaae4d358afdf64f]) successfully deleted
I20260808 13:10:03.608855 1341 ts_tablet_manager.cc:1939] T 6a2ddbefacda4fe48f15fd3d448e7277 P f1cbe5adab9d41889c55c5edecb2360f: Deleting consensus metadata
I20260808 13:10:03.609772 1340 ts_tablet_manager.cc:1905] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:03.610044 720 catalog_manager.cc:4841] TS f1cbe5adab9d41889c55c5edecb2360f (127.31.59.130:33229): tablet 6a2ddbefacda4fe48f15fd3d448e7277 (table test-table [id=5b6e8623113b44a8aaae4d358afdf64f]) successfully deleted
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:03.619048 1340 ts_tablet_manager.cc:1918] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:03.619256 1340 log.cc:1196] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/6a2ddbefacda4fe48f15fd3d448e7277
I20260808 13:10:03.620033 1340 ts_tablet_manager.cc:1939] T 6a2ddbefacda4fe48f15fd3d448e7277 P 728996d21cbd412c8be23b0837dec4d8: Deleting consensus metadata
I20260808 13:10:03.621227 719 catalog_manager.cc:4841] TS 728996d21cbd412c8be23b0837dec4d8 (127.31.59.129:40329): tablet 6a2ddbefacda4fe48f15fd3d448e7277 (table test-table [id=5b6e8623113b44a8aaae4d358afdf64f]) successfully deleted
I20260808 13:10:03.626197 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.589907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.31.59.129:40329)
0808 13:10:03.602996 (+ 13089us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.603003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.603040 (+ 37us) server_negotiation.cc:407] Connection header received
0808 13:10:03.603104 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.603109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.603211 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.603353 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.606661 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.608833 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.611216 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.611928 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.612068 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.617830 (+ 5762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.617843 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.617857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.617921 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.622565 (+ 4644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.623497 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.623505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.623526 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.623713 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.624134 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.624138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.624140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.624663 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.625139 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.625336 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.625617 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12930,"thread_start_us":13830,"threads_started":1}
W20260808 13:10:03.627144 955 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33730
I20260808 13:10:03.650337 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.630601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47750 (local address 127.31.59.190:46777)
0808 13:10:03.630846 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.630851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.630985 (+ 134us) server_negotiation.cc:407] Connection header received
0808 13:10:03.631172 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.631176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.631239 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.631312 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.633801 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.635033 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.637531 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.638284 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.638383 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.642812 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.642818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.642821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.642856 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.646892 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.647819 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.647823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.647825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.648009 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.648416 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.648419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.648421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.648828 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.649114 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.649602 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.649844 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 13:10:03.651257 734 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47750
I20260808 13:10:03.654908 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 813
I20260808 13:10:03.684369 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.655736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.31.59.129:40329)
0808 13:10:03.656047 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.656055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.665991 (+ 9936us) server_negotiation.cc:407] Connection header received
0808 13:10:03.666219 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.666223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.666296 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.666383 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.668202 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.668965 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.670443 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.671268 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.671392 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.675019 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.675025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.675030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.675074 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.680078 (+ 5004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.681088 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.681094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.681098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.681271 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.681917 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.681922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.681926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.682495 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.682869 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.683552 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.683826 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260808 13:10:03.685213 955 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33732
W20260808 13:10:03.686133 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 813: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:03.686426 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 813
I20260808 13:10:03.708245 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 965
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:03.727640 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.709173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.31.59.130:33229)
0808 13:10:03.710353 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.710358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.710377 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:10:03.710433 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.710437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.710496 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.710584 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.713267 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.714231 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.716024 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.716442 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.716530 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.721739 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.721757 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.721773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.721812 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.724648 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.725546 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.725555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.725557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.725731 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.726140 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.726143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.726145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.726544 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.726775 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.727117 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.727299 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":831,"threads_started":1}
W20260808 13:10:03.728382 1104 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39098
W20260808 13:10:03.729076 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 965: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:03.729269 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 965
I20260808 13:10:03.747169 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1113
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:03.763649 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:03.748058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.31.59.131:35893)
0808 13:10:03.748275 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:03.748282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:03.748305 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:10:03.748445 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:03.748451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:03.748556 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:03.748675 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:03.749929 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.751104 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.752371 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:03.753096 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:03.753225 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:03.757638 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:03.757657 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:03.757672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:03.757714 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:03.760710 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.761628 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.761636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.761638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.761782 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:03.762177 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:03.762181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:03.762185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:03.762582 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:03.762824 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:03.763122 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 13:10:03.763328 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 13:10:03.764312 1250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58594
W20260808 13:10:03.765008 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1113: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:03.765192 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 1113
I20260808 13:10:03.786948 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 700
I20260808 13:10:03.788129 777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:03.905551 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 700
2026-08-08T13:10:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4914 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[1353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[1353](info): set up 2 sockets
Aug 08 13:10:03 dist-test-slave-49h3 krb5kdc[1353](info): commencing operation
krb5kdc: starting...
W20260808 13:10:04.969288 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.007s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:05 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:05.166581 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44009
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:05.539866 1369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:05.540263 1369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:05.540480 1369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:05.551700 1369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:05.551893 1369 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:10:05.552007 1369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:05.552135 1369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:05.552244 1369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:05.552402 1369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:05.566807 1369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:44009
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:05.568208 1369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:05.569859 1369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:05.586629 1376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:05.587392 1377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:05.587721 1379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:05.589210 1369 server_base.cc:1034] running on GCE node
I20260808 13:10:05.590238 1369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:05.591794 1369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:05.593133 1369 hybrid_clock.cc:648] HybridClock initialized: now 1786194605592970 us; error 32 us; skew 500 ppm
I20260808 13:10:05.593592 1369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:05 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194605, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:05.598238 1369 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:05.600339 1369 webserver.cc:460] Webserver started at http://127.31.59.190:36761/ using document root <none> and password file <none>
I20260808 13:10:05.601234 1369 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:05.601356 1369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:05.601763 1369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:05.605703 1369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "8b7c9a91983c41b89de96652a08145e0"
format_stamp: "Formatted at 2026-08-08 13:10:05 on dist-test-slave-49h3"
server_key: "584aa421420d32f8a0f841687a3887f1"
server_key_iv: "67c2b213d4a318dd3900abb733bf06bf"
server_key_version: "encryptionkey@0"
I20260808 13:10:05.606709 1369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "8b7c9a91983c41b89de96652a08145e0"
format_stamp: "Formatted at 2026-08-08 13:10:05 on dist-test-slave-49h3"
server_key: "584aa421420d32f8a0f841687a3887f1"
server_key_iv: "67c2b213d4a318dd3900abb733bf06bf"
server_key_version: "encryptionkey@0"
I20260808 13:10:05.613521 1369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260808 13:10:05.618502 1387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:05.619815 1369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:10:05.620018 1369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "8b7c9a91983c41b89de96652a08145e0"
format_stamp: "Formatted at 2026-08-08 13:10:05 on dist-test-slave-49h3"
server_key: "584aa421420d32f8a0f841687a3887f1"
server_key_iv: "67c2b213d4a318dd3900abb733bf06bf"
server_key_version: "encryptionkey@0"
I20260808 13:10:05.620262 1369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:05.638499 1369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:05.644877 1369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:05.645252 1369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:05.669310 1369 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:44009
I20260808 13:10:05.669329 1456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:44009 every 8 connection(s)
I20260808 13:10:05.670709 1369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:05.674474 1457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:05.677814 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 1369
I20260808 13:10:05.678027 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:05.678802 31982 external_mini_cluster.cc:1428] Setting key 72608e0b682718d28ad26b425012addb
I20260808 13:10:05.687433 1457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Bootstrap starting.
I20260808 13:10:05.691255 1457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:05.692566 1457 log.cc:824] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:10:05 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:05.695375 1457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: No bootstrap required, opened a new log
I20260808 13:10:05.701574 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:05.680896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.31.59.190:44009)
0808 13:10:05.681714 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:05.681726 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:05.681769 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:10:05.683466 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:05.683488 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:05.683875 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:05.684201 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:05.685932 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:05.687210 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:05.688930 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:05.689662 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:05.689787 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:05.694276 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:05.694284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:05.694299 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:05.694333 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:05.697280 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:05.698145 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:05.698155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:05.698161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:05.698339 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:05.698702 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:05.698706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:05.698708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:05.699268 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:05.699579 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:05.700002 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 13:10:05.700430 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":394,"threads_started":1}
I20260808 13:10:05.703835 1457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } }
I20260808 13:10:05.704264 1457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:05.704398 1457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7c9a91983c41b89de96652a08145e0, State: Initialized, Role: FOLLOWER
I20260808 13:10:05.704960 1457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } }
I20260808 13:10:05.705202 1457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:05.705401 1457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:05.705621 1457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:05.708053 1457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } }
I20260808 13:10:05.708518 1457 leader_election.cc:302] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b7c9a91983c41b89de96652a08145e0; no voters:
I20260808 13:10:05.709156 1457 leader_election.cc:288] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:05.709331 1462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:05.710088 1462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [term 1 LEADER]: Becoming Leader. State: Replica: 8b7c9a91983c41b89de96652a08145e0, State: Running, Role: LEADER
I20260808 13:10:05.710580 1462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } }
I20260808 13:10:05.711400 1457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:05.714154 1464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b7c9a91983c41b89de96652a08145e0. Latest consensus state: current_term: 1 leader_uuid: "8b7c9a91983c41b89de96652a08145e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } } }
I20260808 13:10:05.714963 1464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:05.714102 1463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b7c9a91983c41b89de96652a08145e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7c9a91983c41b89de96652a08145e0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44009 } } }
I20260808 13:10:05.717649 1463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:05.727425 1470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:05.731776 1470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:05.740989 1470 catalog_manager.cc:1347] Generated new cluster ID: e4d37dcb4e3a4e898d8723b500080980
I20260808 13:10:05.741134 1470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:05.774242 1470 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:05.775641 1470 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:05.784045 1470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Generated new TSK 0
I20260808 13:10:05.784880 1470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:05.841126 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44009
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:06.167582 1485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:06.167900 1485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:06.168040 1485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:06.175343 1485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:06.175485 1485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:06.175623 1485 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:10:06.175693 1485 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:06.175802 1485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:06.185843 1485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:06.187217 1485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:06.188562 1485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:06.201715 1491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:06.201808 1493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:06.201803 1490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:06.202358 1485 server_base.cc:1034] running on GCE node
I20260808 13:10:06.203047 1485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:06.204006 1485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:06.205207 1485 hybrid_clock.cc:648] HybridClock initialized: now 1786194606205154 us; error 41 us; skew 500 ppm
I20260808 13:10:06.205567 1485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:06 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194606, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:06.208678 1485 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:06.210216 1485 webserver.cc:460] Webserver started at http://127.31.59.129:36873/ using document root <none> and password file <none>
I20260808 13:10:06.210892 1485 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:06.211014 1485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:06.211323 1485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:06.214493 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "9b06844055224680b7ca4afa21e813ff"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "6f90fd92c3b4ede0f9f94b63faf42690"
server_key_iv: "97f0d9d2e9568c16f61f49c964bf9d0c"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.215255 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "9b06844055224680b7ca4afa21e813ff"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "6f90fd92c3b4ede0f9f94b63faf42690"
server_key_iv: "97f0d9d2e9568c16f61f49c964bf9d0c"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.220371 1485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:10:06.223867 1500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:06.225324 1485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:10:06.225508 1485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "9b06844055224680b7ca4afa21e813ff"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "6f90fd92c3b4ede0f9f94b63faf42690"
server_key_iv: "97f0d9d2e9568c16f61f49c964bf9d0c"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.225711 1485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:06.237540 1485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:06.241375 1485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:06.241683 1485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:06.244233 1485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:06.244333 1485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:06.244441 1485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:06.244516 1485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:06 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194606, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:06.269598 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.245552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.129:55825)
0808 13:10:06.246442 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:06.246488 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:06.248354 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:06.249296 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:06.249311 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:06.250304 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:06.251325 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.251355 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.252811 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.252817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.255313 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.255336 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.256485 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.256496 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.256788 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.257623 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:06.257650 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:06.260867 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:06.264441 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.264453 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.264473 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.265211 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.265677 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.265684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.265686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.265928 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.266561 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:06.266569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:06.267083 (+ 514us) client_negotiation.cc:769] Sending connection context
0808 13:10:06.267380 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 13:10:06.267972 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":377,"threads_started":1}
I20260808 13:10:06.270382 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.244899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55825 (local address 127.31.59.190:44009)
0808 13:10:06.245415 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:06.245422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:06.246621 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 13:10:06.248636 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:06.248642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:06.248739 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:06.248855 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:06.251553 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.252618 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.255489 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.256006 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.256141 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.261128 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:06.261134 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:06.261137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:06.261173 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:06.264254 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.265375 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.265381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.265385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.265555 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.266061 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.266066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.266070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.266436 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:06.266587 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:06.269668 (+ 3081us) server_negotiation.cc:299] Negotiation successful
0808 13:10:06.269958 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":253,"threads_started":1}
I20260808 13:10:06.289440 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.275804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:45233 (local address 127.31.59.190:44009)
0808 13:10:06.275980 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:06.275984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:06.276803 (+ 819us) server_negotiation.cc:407] Connection header received
0808 13:10:06.277012 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:06.277018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:06.277111 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:06.277241 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:06.278455 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.279183 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.281024 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.281615 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.281731 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.283665 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:06.283671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:06.283676 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:06.283711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:06.286385 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.287252 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.287257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.287261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.287421 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.287828 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.287833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.287836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.288152 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:06.288310 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:06.288882 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 13:10:06.289106 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:10:06.289618 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.275896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.129:45233)
0808 13:10:06.276666 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:06.276691 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:06.276873 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:06.277262 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:06.277265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:06.277842 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:06.278304 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.278316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.279338 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.279346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.280829 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.280849 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.281829 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.281836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.281966 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.282612 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:06.282628 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:06.283482 (+ 854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:06.286510 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.286515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.286520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.287087 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.287516 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.287521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.287524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.287672 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.288380 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:06.288388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:06.288597 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 13:10:06.288751 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 13:10:06.289100 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":421,"threads_started":1}
I20260808 13:10:06.293740 1485 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45777
I20260808 13:10:06.293746 1629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45777 every 8 connection(s)
I20260808 13:10:06.294975 1485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:06.296841 1630 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44009
I20260808 13:10:06.297045 1630 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:06.297689 1630 heartbeater.cc:511] Master 127.31.59.190:44009 requested a full tablet report, sending...
I20260808 13:10:06.299503 1406 ts_manager.cc:194] Registered new tserver with Master: 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777)
I20260808 13:10:06.300913 1406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:55825
I20260808 13:10:06.303718 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 1485
I20260808 13:10:06.303908 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:06.304520 31982 external_mini_cluster.cc:1428] Setting key 45bad7b8e99ec7cad3d36149d0de0cba
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:06.346101 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44009
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:06.662768 1634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:06.663060 1634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:06.663211 1634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:06.670549 1634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:06.670692 1634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:06.670818 1634 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:10:06.670886 1634 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:06.670992 1634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:06.680980 1634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:06.682348 1634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:06.683672 1634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:06.696197 1642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:06.696290 1640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:06.696233 1639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:06.702555 1634 server_base.cc:1034] running on GCE node
I20260808 13:10:06.703213 1634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:06.704214 1634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:06.705418 1634 hybrid_clock.cc:648] HybridClock initialized: now 1786194606705359 us; error 58 us; skew 500 ppm
I20260808 13:10:06.705745 1634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:06 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194606, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:06.708783 1634 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:06.710353 1634 webserver.cc:460] Webserver started at http://127.31.59.130:36921/ using document root <none> and password file <none>
I20260808 13:10:06.711045 1634 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:06.711165 1634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:06.711500 1634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:06.714648 1634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "b4306c776c4945459811253878fa8eae"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "0dd2f650e339f4b8d8bf095c5739d824"
server_key_iv: "9f79620e894d0d2661b9fa6d884ed5e9"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.715421 1634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "b4306c776c4945459811253878fa8eae"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "0dd2f650e339f4b8d8bf095c5739d824"
server_key_iv: "9f79620e894d0d2661b9fa6d884ed5e9"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.720484 1634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:10:06.723927 1649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:06.725322 1634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:06.725504 1634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "b4306c776c4945459811253878fa8eae"
format_stamp: "Formatted at 2026-08-08 13:10:06 on dist-test-slave-49h3"
server_key: "0dd2f650e339f4b8d8bf095c5739d824"
server_key_iv: "9f79620e894d0d2661b9fa6d884ed5e9"
server_key_version: "encryptionkey@0"
I20260808 13:10:06.725696 1634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:06.745688 1634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:06.750046 1634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:06.750306 1634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:06.752884 1634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:06.752995 1634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:06.753115 1634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:06.753186 1634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:06 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194606, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:06.783110 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.753682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:55507 (local address 127.31.59.190:44009)
0808 13:10:06.753881 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:06.753886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:06.755327 (+ 1441us) server_negotiation.cc:407] Connection header received
0808 13:10:06.758202 (+ 2875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:06.758208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:06.758295 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:06.758409 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:06.760677 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.761872 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.764301 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.764912 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.765059 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.776123 (+ 11064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:06.776129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:06.776133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:06.776174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:06.779482 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.780485 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.780491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.780495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.780668 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.781255 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.781258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.781260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.781615 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:06.781775 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:06.782531 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 13:10:06.782773 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:10:06.784262 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.754235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.130:55507)
0808 13:10:06.755113 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:06.755165 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:06.757816 (+ 2651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:06.758693 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:06.758701 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:06.759549 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:06.760439 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.760463 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.762076 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.762082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.764109 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.764129 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.765206 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.765215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.765638 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.772363 (+ 6725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:06.772402 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:06.775829 (+ 3427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:06.779663 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.779670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.779689 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.780353 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.780811 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.780818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.780822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.781132 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.781770 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:06.781779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:06.782176 (+ 397us) client_negotiation.cc:769] Sending connection context
0808 13:10:06.782431 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 13:10:06.783004 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":344,"threads_started":1}
I20260808 13:10:06.798704 1634 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:45713
I20260808 13:10:06.798746 1783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:45713 every 8 connection(s)
I20260808 13:10:06.800026 1634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:06.802223 1784 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44009
I20260808 13:10:06.802969 1784 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:06.803772 1784 heartbeater.cc:511] Master 127.31.59.190:44009 requested a full tablet report, sending...
I20260808 13:10:06.804162 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 1634
I20260808 13:10:06.804324 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:06.804879 31982 external_mini_cluster.cc:1428] Setting key 27f8dc7ac913de92f29523767d13f20e
I20260808 13:10:06.805327 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.790425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:44467 (local address 127.31.59.190:44009)
0808 13:10:06.790609 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:06.790615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:06.791580 (+ 965us) server_negotiation.cc:407] Connection header received
0808 13:10:06.791874 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:06.791880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:06.791962 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:06.792038 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:06.793742 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.794918 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.797055 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.797493 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.797615 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.799651 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:06.799656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:06.799660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:06.799691 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:06.802632 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.803309 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.803315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.803317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.803488 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:06.803835 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:06.803840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:06.803843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:06.804289 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:06.804443 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:06.804909 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 13:10:06.805108 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:10:06.805660 1770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:06.790492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.130:44467)
0808 13:10:06.791470 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:06.791491 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:06.791709 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:06.792279 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:06.792284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:06.793016 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:06.793560 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.793574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.795109 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.795115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.796926 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:06.796940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:06.797699 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:06.797706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:06.797922 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:06.798617 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:06.798636 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:06.799471 (+ 835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:06.802729 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.802733 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.802736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.803197 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.803599 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:06.803602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:06.803604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:06.803721 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:06.804461 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:06.804466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:06.804652 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 13:10:06.804814 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 13:10:06.805119 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":666,"threads_started":1}
I20260808 13:10:06.806175 1406 ts_manager.cc:194] Registered new tserver with Master: b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:06.807135 1406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:55507
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:06.846132 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44009
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:07.166965 1788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:07.167275 1788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:07.167424 1788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:07.174671 1788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:07.174800 1788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:07.174923 1788 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:10:07.174991 1788 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:07.175102 1788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:07.185101 1788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:07.186602 1788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:07.187942 1788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:07.200765 1797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:07.200717 1795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:07.200693 1793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:07.201531 1788 server_base.cc:1034] running on GCE node
I20260808 13:10:07.202147 1788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:07.203075 1788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:07.204278 1788 hybrid_clock.cc:648] HybridClock initialized: now 1786194607204222 us; error 44 us; skew 500 ppm
I20260808 13:10:07.204553 1788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:07 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194607, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:07.207554 1788 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:07.208926 1788 webserver.cc:460] Webserver started at http://127.31.59.131:38869/ using document root <none> and password file <none>
I20260808 13:10:07.209620 1788 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:07.209724 1788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:07.209986 1788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:07.213065 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "2434f89e87e646ed89621e3c7721ae3d"
format_stamp: "Formatted at 2026-08-08 13:10:07 on dist-test-slave-49h3"
server_key: "a3583bb21c768983d0dfcab7ddaca5b5"
server_key_iv: "f9a5a8d833ca6ce1577a15177b58264c"
server_key_version: "encryptionkey@0"
I20260808 13:10:07.213917 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "2434f89e87e646ed89621e3c7721ae3d"
format_stamp: "Formatted at 2026-08-08 13:10:07 on dist-test-slave-49h3"
server_key: "a3583bb21c768983d0dfcab7ddaca5b5"
server_key_iv: "f9a5a8d833ca6ce1577a15177b58264c"
server_key_version: "encryptionkey@0"
I20260808 13:10:07.218825 1788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:10:07.222083 1806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:07.223439 1788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:07.223572 1788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "2434f89e87e646ed89621e3c7721ae3d"
format_stamp: "Formatted at 2026-08-08 13:10:07 on dist-test-slave-49h3"
server_key: "a3583bb21c768983d0dfcab7ddaca5b5"
server_key_iv: "f9a5a8d833ca6ce1577a15177b58264c"
server_key_version: "encryptionkey@0"
I20260808 13:10:07.223762 1788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:07.246178 1788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:07.250023 1788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:07.250275 1788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:07.252774 1788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:07.252897 1788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:07.253010 1788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:07.253093 1788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:07 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194607, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:07.274896 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.253462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:44235 (local address 127.31.59.190:44009)
0808 13:10:07.253670 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.253676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.255109 (+ 1433us) server_negotiation.cc:407] Connection header received
0808 13:10:07.257057 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.257063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.257151 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.257240 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:07.259456 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.260629 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.262988 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.263568 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.263682 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.268271 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:07.268276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:07.268279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:07.268312 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:07.271155 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.272138 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.272144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.272148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.272416 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.272870 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.272876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.272880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.273284 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:07.273514 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.274214 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.274483 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:10:07.275604 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.254159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.131:44235)
0808 13:10:07.254930 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.254975 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.256775 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.257551 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.257559 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.258317 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:07.259192 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.259215 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.260806 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.260816 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.262835 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.262848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.263830 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.263839 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.264073 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.264952 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:07.264996 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:07.268058 (+ 3062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:07.271323 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.271332 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.271353 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.271984 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.272530 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.272535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.272540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.272734 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.273426 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:07.273434 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:07.273826 (+ 392us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.274047 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.274426 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":331,"threads_started":1}
I20260808 13:10:07.297544 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.281515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:36913 (local address 127.31.59.190:44009)
0808 13:10:07.281724 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.281731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.282530 (+ 799us) server_negotiation.cc:407] Connection header received
0808 13:10:07.282714 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.282718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.282784 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.282873 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:07.284300 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.285441 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.287335 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.288075 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.288213 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.290223 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:07.290230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:07.290235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:07.290281 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:07.294367 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.295223 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.295229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.295233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.295401 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.295725 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.295737 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.295741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.296091 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:07.296260 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.296907 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.297162 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:10:07.297546 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.281402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.31.59.131:36913)
0808 13:10:07.282429 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.282452 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.282605 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.282932 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.282937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.283737 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:07.284109 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.284122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.285627 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.285633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.287155 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.287169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.288264 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.288268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.288391 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.288990 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:07.289008 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:07.289982 (+ 974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:07.294527 (+ 4545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.294533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.294537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.295100 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.295509 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.295512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.295515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.295640 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.296198 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:07.296201 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:07.296430 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.296599 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.296978 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":690,"threads_started":1}
I20260808 13:10:07.303522 1630 heartbeater.cc:503] Master 127.31.59.190:44009 was elected leader, sending a full tablet report...
I20260808 13:10:07.309645 1788 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:46431
I20260808 13:10:07.309650 1941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:46431 every 8 connection(s)
I20260808 13:10:07.310921 1788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:07.312820 1942 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44009
I20260808 13:10:07.313036 1942 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:07.313716 1942 heartbeater.cc:511] Master 127.31.59.190:44009 requested a full tablet report, sending...
I20260808 13:10:07.314401 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 1788
I20260808 13:10:07.314599 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:07.315095 31982 external_mini_cluster.cc:1428] Setting key 89721198365ca3a9faf5e09df7868f9f
I20260808 13:10:07.315162 1406 ts_manager.cc:194] Registered new tserver with Master: 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:07.316152 1406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:44235
I20260808 13:10:07.317972 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:10:07 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:10:07 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:07.366578 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.353375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.31.59.190:44009)
0808 13:10:07.353646 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.353652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.353886 (+ 234us) server_negotiation.cc:407] Connection header received
0808 13:10:07.354054 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.354058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.354126 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.354205 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:07.355622 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.356384 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.357548 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.357975 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.358067 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.361683 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:07.361688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:07.361691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:07.361730 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:07.364254 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.364804 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.364808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.364810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.364943 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.365235 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.365238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.365240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.365579 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:07.365762 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.366007 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.366202 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 13:10:07.370622 1406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:07.377667 1406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:07.399219 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.389695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.31.59.129:45777)
0808 13:10:07.390241 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.390250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.390383 (+ 133us) server_negotiation.cc:407] Connection header received
0808 13:10:07.390721 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.390730 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.390984 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.391850 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.392078 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.393731 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.396550 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.398076 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.398207 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.398417 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.398763 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.398925 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":247,"threads_started":1}
I20260808 13:10:07.399354 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.389511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45777 (local address 127.0.0.1:49262)
0808 13:10:07.390247 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.390296 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.390471 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.391213 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.391221 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.391270 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.391637 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.391650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.393869 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.393874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.396377 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.396392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.398311 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.398317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.398507 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.398558 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.398695 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.398879 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":4,"thread_start_us":345,"threads_started":1}
I20260808 13:10:07.399755 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.390526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:45713 (local address 127.0.0.1:32836)
0808 13:10:07.391057 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.391075 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.391186 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.392101 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.392106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.392133 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.392531 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.392542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.393920 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.393924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.396901 (+ 2977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.396912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.398999 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.399008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.399175 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.399209 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.399273 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.399408 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":971,"thread_start_us":269,"threads_started":1}
I20260808 13:10:07.400477 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.390808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.31.59.130:45713)
0808 13:10:07.391649 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.391655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.391675 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:07.391753 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.391757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.391900 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.392693 (+ 793us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.392849 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.393777 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.397053 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.398766 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.399728 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.399982 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.400041 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.400194 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":580,"threads_started":1}
I20260808 13:10:07.402072 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.391119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.0.0.1:33510)
0808 13:10:07.394371 (+ 3252us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.394391 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.394503 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.395167 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.395172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.395200 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.395680 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.395690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.397432 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.397438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.399431 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.399444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.401294 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.401301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.401479 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.401511 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.401595 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.401717 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3131,"mutex_wait_us":2,"thread_start_us":2939,"threads_started":1}
I20260808 13:10:07.402329 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.391677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.31.59.131:46431)
0808 13:10:07.392438 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.392446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.394628 (+ 2182us) server_negotiation.cc:407] Connection header received
0808 13:10:07.394738 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.394745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.394991 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.395182 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.395892 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.397268 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.399621 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.400805 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.400924 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.401142 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.401922 (+ 780us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.402038 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":400,"threads_started":1}
I20260808 13:10:07.403906 1556 tablet_service.cc:1467] Processing CreateTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 (DEFAULT_TABLE table=test-table [id=1eb098414caf4135bce3315f3add49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:07.404654 1705 tablet_service.cc:1467] Processing CreateTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 (DEFAULT_TABLE table=test-table [id=1eb098414caf4135bce3315f3add49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:07.404855 1863 tablet_service.cc:1467] Processing CreateTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 (DEFAULT_TABLE table=test-table [id=1eb098414caf4135bce3315f3add49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:07.405392 1556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d40092c2ea4433a08ff1ac88a439e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:07.405941 1705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d40092c2ea4433a08ff1ac88a439e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:07.406229 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d40092c2ea4433a08ff1ac88a439e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:07.417678 1962 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Bootstrap starting.
I20260808 13:10:07.420830 1962 tablet_bootstrap.cc:654] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:07.421692 1963 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Bootstrap starting.
I20260808 13:10:07.422137 1962 log.cc:824] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:07.422605 1964 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Bootstrap starting.
I20260808 13:10:07.425694 1963 tablet_bootstrap.cc:654] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:07.426407 1964 tablet_bootstrap.cc:654] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:07.426935 1963 log.cc:824] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:07.427577 1964 log.cc:824] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:07.428468 1962 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: No bootstrap required, opened a new log
I20260808 13:10:07.428754 1962 ts_tablet_manager.cc:1397] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260808 13:10:07.430655 1964 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: No bootstrap required, opened a new log
I20260808 13:10:07.430917 1964 ts_tablet_manager.cc:1397] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 13:10:07.431067 1963 tablet_bootstrap.cc:492] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: No bootstrap required, opened a new log
I20260808 13:10:07.431299 1963 ts_tablet_manager.cc:1397] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 13:10:07.438525 1963 raft_consensus.cc:348] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.438896 1963 raft_consensus.cc:374] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:07.438987 1963 raft_consensus.cc:729] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4306c776c4945459811253878fa8eae, State: Initialized, Role: FOLLOWER
I20260808 13:10:07.438959 1962 raft_consensus.cc:348] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.439366 1962 raft_consensus.cc:374] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:07.439461 1962 raft_consensus.cc:729] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Initialized, Role: FOLLOWER
I20260808 13:10:07.439558 1963 consensus_queue.cc:260] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.440001 1962 consensus_queue.cc:260] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.440716 1784 heartbeater.cc:503] Master 127.31.59.190:44009 was elected leader, sending a full tablet report...
I20260808 13:10:07.441118 1963 ts_tablet_manager.cc:1428] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260808 13:10:07.442535 1964 raft_consensus.cc:348] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.443007 1964 raft_consensus.cc:374] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:07.443169 1964 raft_consensus.cc:729] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Initialized, Role: FOLLOWER
I20260808 13:10:07.443971 1964 consensus_queue.cc:260] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.446722 1964 ts_tablet_manager.cc:1428] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260808 13:10:07.447856 1942 heartbeater.cc:503] Master 127.31.59.190:44009 was elected leader, sending a full tablet report...
I20260808 13:10:07.448282 1962 ts_tablet_manager.cc:1428] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
W20260808 13:10:07.547149 1631 tablet.cc:2369] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:07.552264 1785 tablet.cc:2369] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:07.562474 1943 tablet.cc:2369] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:07.825841 1971 raft_consensus.cc:482] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:07.826143 1971 raft_consensus.cc:504] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.827044 1972 raft_consensus.cc:482] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:07.827306 1972 raft_consensus.cc:504] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.827873 1971 leader_election.cc:288] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:07.828943 1972 leader_election.cc:288] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4306c776c4945459811253878fa8eae (127.31.59.130:45713), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:07.836504 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.829041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:54321 (local address 127.31.59.130:45713)
0808 13:10:07.829224 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.829241 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.830084 (+ 843us) server_negotiation.cc:407] Connection header received
0808 13:10:07.830302 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.830307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.830372 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.830445 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.830901 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.831935 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.834405 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835478 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.835612 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.835676 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.836146 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.836265 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 13:10:07.836524 1973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.828141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45777 (local address 127.31.59.131:38633)
0808 13:10:07.828699 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.828717 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.828866 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.829479 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.829484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.829513 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.829822 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.829839 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.831366 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.831371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.833928 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.833939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835281 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.835288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.835501 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.835535 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.836123 (+ 588us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.836231 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":355,"thread_start_us":327,"threads_started":1}
I20260808 13:10:07.836637 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.828883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:45713 (local address 127.31.59.131:54321)
0808 13:10:07.829989 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.830004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.830107 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.830478 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.830482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.830506 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.830753 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.830761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.832099 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.832104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.834194 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.834227 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835824 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.835830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.836005 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.836036 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.836119 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.836260 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":50,"thread_start_us":790,"threads_started":1}
I20260808 13:10:07.836766 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.828296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:38633 (local address 127.31.59.129:45777)
0808 13:10:07.828487 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.828493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.829148 (+ 655us) server_negotiation.cc:407] Connection header received
0808 13:10:07.829259 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.829263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.829322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.829410 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.830047 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.831204 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.834092 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835129 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.835281 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.835354 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.835683 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.835824 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 13:10:07.837874 1728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae" is_pre_election: true
I20260808 13:10:07.838037 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff" is_pre_election: true
I20260808 13:10:07.838498 1577 raft_consensus.cc:2454] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:07.838501 1728 raft_consensus.cc:2454] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:07.839232 1808 leader_election.cc:302] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, 9b06844055224680b7ca4afa21e813ff; no voters:
I20260808 13:10:07.839687 1971 raft_consensus.cc:2781] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:07.839841 1971 raft_consensus.cc:482] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:07.840034 1971 raft_consensus.cc:3037] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:07.840349 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.829672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43369 (local address 127.31.59.130:45713)
0808 13:10:07.830353 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.830358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.830941 (+ 583us) server_negotiation.cc:407] Connection header received
0808 13:10:07.831103 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.831107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.831172 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.831276 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.832109 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.833242 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.837692 (+ 4450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.839254 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.839403 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.839468 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.840002 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.840112 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":690,"threads_started":1}
I20260808 13:10:07.840884 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.829340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:45713 (local address 127.31.59.129:43369)
0808 13:10:07.830857 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.830874 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.830992 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.831525 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.831530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.831555 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.831922 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.831932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.833430 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.833435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.837516 (+ 4081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.837530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.839653 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.839660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.839823 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.839855 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.840493 (+ 638us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.840599 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":1087,"thread_start_us":1732,"threads_started":1}
I20260808 13:10:07.841434 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.831763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:51533 (local address 127.31.59.131:46431)
0808 13:10:07.831933 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.831938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.833515 (+ 1577us) server_negotiation.cc:407] Connection header received
0808 13:10:07.833774 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.833779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.833863 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.833923 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:07.834669 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835685 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.839017 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.840288 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.840417 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.840479 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.841052 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.841188 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 13:10:07.841420 1728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae" is_pre_election: true
I20260808 13:10:07.841646 1728 raft_consensus.cc:2454] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b06844055224680b7ca4afa21e813ff in term 0.
I20260808 13:10:07.842103 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.831445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.31.59.129:51533)
0808 13:10:07.833409 (+ 1964us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.833427 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.833613 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.834091 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.834096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.834119 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:07.834514 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.834526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.835856 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.835861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.838832 (+ 2971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.838847 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.840676 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.840682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.840871 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.840906 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.841594 (+ 688us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.841726 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1852,"mutex_wait_us":23,"thread_start_us":1695,"threads_started":1}
I20260808 13:10:07.842056 1502 leader_election.cc:302] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b06844055224680b7ca4afa21e813ff, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:07.842365 1972 raft_consensus.cc:2781] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:07.842494 1972 raft_consensus.cc:482] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:07.842571 1972 raft_consensus.cc:3037] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:07.842932 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d" is_pre_election: true
I20260808 13:10:07.843504 1971 raft_consensus.cc:504] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.843978 1891 raft_consensus.cc:2379] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b06844055224680b7ca4afa21e813ff in current term 1: Already voted for candidate 2434f89e87e646ed89621e3c7721ae3d in this term.
I20260808 13:10:07.844187 1971 leader_election.cc:288] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Requested vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:07.844859 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff"
I20260808 13:10:07.844907 1728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae"
I20260808 13:10:07.845099 1728 raft_consensus.cc:3037] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:07.846498 1972 raft_consensus.cc:504] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.846951 1577 raft_consensus.cc:2379] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2434f89e87e646ed89621e3c7721ae3d in current term 1: Already voted for candidate 9b06844055224680b7ca4afa21e813ff in this term.
I20260808 13:10:07.847226 1972 leader_election.cc:288] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 election: Requested vote from peers b4306c776c4945459811253878fa8eae (127.31.59.130:45713), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:07.847729 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d"
I20260808 13:10:07.847973 1891 raft_consensus.cc:2379] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b06844055224680b7ca4afa21e813ff in current term 1: Already voted for candidate 2434f89e87e646ed89621e3c7721ae3d in this term.
I20260808 13:10:07.848068 1728 raft_consensus.cc:2454] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 1.
I20260808 13:10:07.848361 1727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d40092c2ea4433a08ff1ac88a439e6" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae"
I20260808 13:10:07.848517 1808 leader_election.cc:302] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, b4306c776c4945459811253878fa8eae; no voters: 9b06844055224680b7ca4afa21e813ff
I20260808 13:10:07.848682 1727 raft_consensus.cc:2379] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b06844055224680b7ca4afa21e813ff in current term 1: Already voted for candidate 2434f89e87e646ed89621e3c7721ae3d in this term.
I20260808 13:10:07.848799 1971 raft_consensus.cc:2781] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:07.849267 1971 raft_consensus.cc:686] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 LEADER]: Becoming Leader. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Running, Role: LEADER
I20260808 13:10:07.849261 1502 leader_election.cc:302] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b06844055224680b7ca4afa21e813ff; no voters: 2434f89e87e646ed89621e3c7721ae3d, b4306c776c4945459811253878fa8eae
I20260808 13:10:07.849570 1972 raft_consensus.cc:2726] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:10:07.849822 1971 consensus_queue.cc:237] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:07.854318 1406 catalog_manager.cc:5495] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131). New cstate: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } health_report { overall_health: HEALTHY } } }
Aug 08 13:10:07 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194605, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:07.996537 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.983887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.31.59.190:44009)
0808 13:10:07.984710 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:07.984714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:07.984785 (+ 71us) server_negotiation.cc:407] Connection header received
0808 13:10:07.984934 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:07.984938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:07.984994 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:07.985086 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:07.986208 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.987015 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.988212 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.988612 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.988700 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.991846 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:07.991851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:07.991854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:07.991883 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:07.994167 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.994814 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.994818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.994820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.994943 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:07.995260 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:07.995263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:07.995265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:07.995524 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:07.995630 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:07.996018 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 13:10:07.996259 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":711,"thread_start_us":460,"threads_started":1}
I20260808 13:10:07.996573 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:07.983875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44009 (local address 127.0.0.1:42540)
0808 13:10:07.984678 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:07.984697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:07.984839 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:07.985114 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:07.985117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:07.985833 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:07.986066 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.986075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.987148 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.987152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.988097 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:07.988108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:07.988755 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:07.988759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:07.988884 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:07.989554 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:07.989571 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:07.991669 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:07.994285 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.994296 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.994302 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.994708 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.995028 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:07.995031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:07.995033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:07.995170 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:07.995623 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:07.995635 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:07.995803 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 13:10:07.995931 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 13:10:07.996140 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":442,"threads_started":1}
I20260808 13:10:08.001555 1406 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:42540:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:10:08.002480 1406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:08.008914 1556 tablet_service.cc:1467] Processing CreateTablet for tablet 0427ac4fffb0414cb3173da2a24f149f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7255a304c3c4646998b698a8479b837]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:08.009007 1863 tablet_service.cc:1467] Processing CreateTablet for tablet 0427ac4fffb0414cb3173da2a24f149f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7255a304c3c4646998b698a8479b837]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:08.009007 1705 tablet_service.cc:1467] Processing CreateTablet for tablet 0427ac4fffb0414cb3173da2a24f149f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7255a304c3c4646998b698a8479b837]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:08.009895 1556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0427ac4fffb0414cb3173da2a24f149f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.009902 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0427ac4fffb0414cb3173da2a24f149f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.009902 1705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0427ac4fffb0414cb3173da2a24f149f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.016139 1990 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: Bootstrap starting.
I20260808 13:10:08.016997 1992 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: Bootstrap starting.
I20260808 13:10:08.018374 1990 tablet_bootstrap.cc:654] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.018478 1991 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: Bootstrap starting.
I20260808 13:10:08.018606 1992 tablet_bootstrap.cc:654] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.019829 1991 tablet_bootstrap.cc:654] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.020680 1990 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: No bootstrap required, opened a new log
I20260808 13:10:08.020836 1990 ts_tablet_manager.cc:1397] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 13:10:08.021257 1992 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: No bootstrap required, opened a new log
I20260808 13:10:08.021413 1992 ts_tablet_manager.cc:1397] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 13:10:08.021654 1990 raft_consensus.cc:348] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.021906 1990 raft_consensus.cc:374] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.022017 1990 raft_consensus.cc:729] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.022183 1991 tablet_bootstrap.cc:492] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: No bootstrap required, opened a new log
I20260808 13:10:08.022334 1991 ts_tablet_manager.cc:1397] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 13:10:08.022181 1992 raft_consensus.cc:348] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.022280 1990 consensus_queue.cc:260] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.022423 1992 raft_consensus.cc:374] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.022516 1992 raft_consensus.cc:729] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.022677 1972 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.022720 1992 consensus_queue.cc:260] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.022914 1972 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.023065 1991 raft_consensus.cc:348] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.023137 1971 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.023291 1992 ts_tablet_manager.cc:1428] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 13:10:08.023309 1991 raft_consensus.cc:374] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.023399 1971 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.023393 1991 raft_consensus.cc:729] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4306c776c4945459811253878fa8eae, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.023586 1991 consensus_queue.cc:260] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.024186 1990 ts_tablet_manager.cc:1428] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 13:10:08.025051 1991 ts_tablet_manager.cc:1428] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:08.025826 1997 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.026289 1997 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.180246 1971 raft_consensus.cc:482] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:08.180495 1971 raft_consensus.cc:504] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.181154 1971 leader_election.cc:288] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:08.181864 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0427ac4fffb0414cb3173da2a24f149f" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff" is_pre_election: true
I20260808 13:10:08.181918 1727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0427ac4fffb0414cb3173da2a24f149f" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae" is_pre_election: true
I20260808 13:10:08.182121 1577 raft_consensus.cc:2454] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:08.182194 1727 raft_consensus.cc:2454] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:08.182574 1808 leader_election.cc:302] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, 9b06844055224680b7ca4afa21e813ff; no voters:
I20260808 13:10:08.182840 1971 raft_consensus.cc:2781] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:08.182924 1971 raft_consensus.cc:482] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:08.183000 1971 raft_consensus.cc:3037] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.185019 1971 raft_consensus.cc:504] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.185555 1971 leader_election.cc:288] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Requested vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:08.186101 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0427ac4fffb0414cb3173da2a24f149f" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff"
I20260808 13:10:08.186208 1727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0427ac4fffb0414cb3173da2a24f149f" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae"
I20260808 13:10:08.186339 1577 raft_consensus.cc:3037] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.186416 1727 raft_consensus.cc:3037] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.189523 1577 raft_consensus.cc:2454] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 1.
I20260808 13:10:08.189523 1727 raft_consensus.cc:2454] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 1.
I20260808 13:10:08.189917 1808 leader_election.cc:302] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:08.190174 1971 raft_consensus.cc:2781] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:08.190352 1971 raft_consensus.cc:686] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [term 1 LEADER]: Becoming Leader. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Running, Role: LEADER
I20260808 13:10:08.190618 1971 consensus_queue.cc:237] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.192236 1979 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: TxnStatusTablet state changed. Reason: New leader 2434f89e87e646ed89621e3c7721ae3d. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.192638 1979 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:08.193301 1999 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:10:08.193574 1999 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:10:08.193710 1405 catalog_manager.cc:5495] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131). New cstate: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } health_report { overall_health: HEALTHY } } }
I20260808 13:10:08.212277 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.206113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.31.59.131:46431)
0808 13:10:08.206322 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.206326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.206400 (+ 74us) server_negotiation.cc:407] Connection header received
0808 13:10:08.206666 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.206669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.206785 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.206894 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.207323 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.208102 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.209999 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.210461 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.210550 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.211149 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.211666 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.211776 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.211930 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.212006 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:10:08.212668 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.206028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.0.0.1:33526)
0808 13:10:08.206309 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:08.206329 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:08.206484 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:08.206955 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:08.206961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:08.206974 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:08.207182 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.207195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.208237 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.208250 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.209865 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.209881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.210617 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.210624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.210876 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.210951 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.211775 (+ 824us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.211800 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:10:08.211889 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 13:10:08.212066 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 13:10:08.220854 1577 raft_consensus.cc:1261] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Refusing update from remote peer 2434f89e87e646ed89621e3c7721ae3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:08.220852 1727 raft_consensus.cc:1261] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Refusing update from remote peer 2434f89e87e646ed89621e3c7721ae3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:08.221678 1971 consensus_queue.cc:1035] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.222074 1979 consensus_queue.cc:1035] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.238948 1972 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: TxnStatusTablet state changed. Reason: New leader 2434f89e87e646ed89621e3c7721ae3d. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.240139 1972 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.240931 1997 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: TxnStatusTablet state changed. Reason: New leader 2434f89e87e646ed89621e3c7721ae3d. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.241232 1997 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.243137 2003 mvcc.cc:205] Tried to move back new op lower bound from 7316253115262177280 to 7316253115150860288. Current Snapshot: MvccSnapshot[applied={T|T < 7316253115262177280}]
I20260808 13:10:08.246057 1971 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.246369 1971 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:08.250385 1979 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.250648 1979 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 2434f89e87e646ed89621e3c7721ae3d: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:08.260608 2008 consensus_queue.cc:1035] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.262687 1972 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.262984 1972 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P 9b06844055224680b7ca4afa21e813ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.267133 1997 tablet_replica.cc:439] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } } }
I20260808 13:10:08.267413 1997 tablet_replica.cc:442] T 0427ac4fffb0414cb3173da2a24f149f P b4306c776c4945459811253878fa8eae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:08.276616 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.266677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.31.59.131:46431)
0808 13:10:08.266877 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.266883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.270319 (+ 3436us) server_negotiation.cc:407] Connection header received
0808 13:10:08.270606 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.270612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.270702 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.270767 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.271440 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.272479 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.274505 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.275153 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.275278 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.275863 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.275991 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.276067 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.276259 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.276339 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:10:08.276652 1979 consensus_queue.cc:1035] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.295930 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.288505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:37051 (local address 127.31.59.131:46431)
0808 13:10:08.288717 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.288723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.289382 (+ 659us) server_negotiation.cc:407] Connection header received
0808 13:10:08.289595 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.289598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.289657 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.289744 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.290352 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.291547 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.293751 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.294444 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.294568 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.295118 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.295271 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.295350 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.295502 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.295600 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":54}
I20260808 13:10:08.296142 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.288577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.31.59.131:37051)
0808 13:10:08.289280 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:08.289300 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:08.289470 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:08.289768 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:08.289773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:08.289787 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:08.290192 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.290205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.291691 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.291699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.293604 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.293629 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.294653 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.294661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.294843 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.294912 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.295372 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.295384 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 13:10:08.295469 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 13:10:08.295626 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":16,"thread_start_us":370,"threads_started":1}
I20260808 13:10:08.379820 1406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:08.380098 1406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:08.383888 1406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Sending DeleteTablet for 3 replicas of tablet 79d40092c2ea4433a08ff1ac88a439e6
I20260808 13:10:08.385030 1556 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:08 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:49262
I20260808 13:10:08.385113 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:08 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33510
I20260808 13:10:08.385259 1705 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d40092c2ea4433a08ff1ac88a439e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:08 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:32836
I20260808 13:10:08.387765 1406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:08.388690 1406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:08.396324 1555 tablet_service.cc:1467] Processing CreateTablet for tablet 929f7c72d4b04e019287048d7569bf7d (DEFAULT_TABLE table=test-table [id=3723fe016868440f9217962d7d4161dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:08.396865 1555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929f7c72d4b04e019287048d7569bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.397927 2024 tablet_replica.cc:330] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: stopping tablet replica
I20260808 13:10:08.398550 2024 raft_consensus.cc:2229] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:08.399307 2024 raft_consensus.cc:2258] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:08.401897 2024 ts_tablet_manager.cc:1905] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:08.403672 1990 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Bootstrap starting.
I20260808 13:10:08.404700 1704 tablet_service.cc:1467] Processing CreateTablet for tablet 929f7c72d4b04e019287048d7569bf7d (DEFAULT_TABLE table=test-table [id=3723fe016868440f9217962d7d4161dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:08.404937 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 929f7c72d4b04e019287048d7569bf7d (DEFAULT_TABLE table=test-table [id=3723fe016868440f9217962d7d4161dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:08.405190 1704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929f7c72d4b04e019287048d7569bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.405308 1990 tablet_bootstrap.cc:654] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.405380 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929f7c72d4b04e019287048d7569bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:08.412007 1991 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Bootstrap starting.
I20260808 13:10:08.413827 1991 tablet_bootstrap.cc:654] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.417048 1992 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Bootstrap starting.
I20260808 13:10:08.418977 1992 tablet_bootstrap.cc:654] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:08.423554 2025 tablet_replica.cc:330] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: stopping tablet replica
I20260808 13:10:08.424748 2025 raft_consensus.cc:2229] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:08.425484 2025 raft_consensus.cc:2258] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:08.428534 1991 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: No bootstrap required, opened a new log
I20260808 13:10:08.428575 2025 ts_tablet_manager.cc:1905] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:08.428817 1991 ts_tablet_manager.cc:1397] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.005s
I20260808 13:10:08.429804 1991 raft_consensus.cc:348] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.430471 1991 raft_consensus.cc:374] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.430604 1991 raft_consensus.cc:729] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4306c776c4945459811253878fa8eae, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.432907 2024 ts_tablet_manager.cc:1918] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:08.433312 1991 consensus_queue.cc:260] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.433789 2024 log.cc:1196] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/79d40092c2ea4433a08ff1ac88a439e6
I20260808 13:10:08.434523 1991 ts_tablet_manager.cc:1428] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 13:10:08.434808 1990 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: No bootstrap required, opened a new log
I20260808 13:10:08.435037 1990 ts_tablet_manager.cc:1397] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.009s
I20260808 13:10:08.435667 2024 ts_tablet_manager.cc:1939] T 79d40092c2ea4433a08ff1ac88a439e6 P 2434f89e87e646ed89621e3c7721ae3d: Deleting consensus metadata
I20260808 13:10:08.436496 1990 raft_consensus.cc:348] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.438535 1990 raft_consensus.cc:374] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.438578 1389 catalog_manager.cc:4841] TS 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431): tablet 79d40092c2ea4433a08ff1ac88a439e6 (table test-table [id=1eb098414caf4135bce3315f3add49f4]) successfully deleted
I20260808 13:10:08.438746 1990 raft_consensus.cc:729] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.439060 1990 consensus_queue.cc:260] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.439704 1990 ts_tablet_manager.cc:1428] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 13:10:08.440883 2023 tablet_replica.cc:330] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: stopping tablet replica
I20260808 13:10:08.441254 2025 ts_tablet_manager.cc:1918] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:08.441423 2025 log.cc:1196] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/79d40092c2ea4433a08ff1ac88a439e6
I20260808 13:10:08.441941 2025 ts_tablet_manager.cc:1939] T 79d40092c2ea4433a08ff1ac88a439e6 P b4306c776c4945459811253878fa8eae: Deleting consensus metadata
I20260808 13:10:08.443035 1992 tablet_bootstrap.cc:492] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: No bootstrap required, opened a new log
I20260808 13:10:08.443274 1992 ts_tablet_manager.cc:1397] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.003s
I20260808 13:10:08.443608 1389 catalog_manager.cc:4841] TS b4306c776c4945459811253878fa8eae (127.31.59.130:45713): tablet 79d40092c2ea4433a08ff1ac88a439e6 (table test-table [id=1eb098414caf4135bce3315f3add49f4]) successfully deleted
I20260808 13:10:08.444018 2023 raft_consensus.cc:2229] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:08.444228 1992 raft_consensus.cc:348] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.444362 2023 raft_consensus.cc:2258] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:08.444455 1992 raft_consensus.cc:374] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:08.444536 1992 raft_consensus.cc:729] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Initialized, Role: FOLLOWER
I20260808 13:10:08.444792 1992 consensus_queue.cc:260] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.445282 1992 ts_tablet_manager.cc:1428] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 13:10:08.446295 2023 ts_tablet_manager.cc:1905] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:08.453817 2023 ts_tablet_manager.cc:1918] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:08.453994 2023 log.cc:1196] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/79d40092c2ea4433a08ff1ac88a439e6
I20260808 13:10:08.454533 2023 ts_tablet_manager.cc:1939] T 79d40092c2ea4433a08ff1ac88a439e6 P 9b06844055224680b7ca4afa21e813ff: Deleting consensus metadata
I20260808 13:10:08.456352 1389 catalog_manager.cc:4841] TS 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777): tablet 79d40092c2ea4433a08ff1ac88a439e6 (table test-table [id=1eb098414caf4135bce3315f3add49f4]) successfully deleted
I20260808 13:10:08.459614 2014 raft_consensus.cc:482] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:08.459815 2014 raft_consensus.cc:504] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.460543 2014 leader_election.cc:288] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4306c776c4945459811253878fa8eae (127.31.59.130:45713), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:08.461129 1727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929f7c72d4b04e019287048d7569bf7d" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae" is_pre_election: true
I20260808 13:10:08.461228 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929f7c72d4b04e019287048d7569bf7d" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d" is_pre_election: true
I20260808 13:10:08.461369 1727 raft_consensus.cc:2454] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b06844055224680b7ca4afa21e813ff in term 0.
I20260808 13:10:08.461604 1891 raft_consensus.cc:2454] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b06844055224680b7ca4afa21e813ff in term 0.
I20260808 13:10:08.461835 1502 leader_election.cc:302] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b06844055224680b7ca4afa21e813ff, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:08.462253 2014 raft_consensus.cc:2781] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:08.462374 2014 raft_consensus.cc:482] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:08.462471 2014 raft_consensus.cc:3037] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.464569 2014 raft_consensus.cc:504] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.465054 2014 leader_election.cc:288] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 election: Requested vote from peers b4306c776c4945459811253878fa8eae (127.31.59.130:45713), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:08.465569 1727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929f7c72d4b04e019287048d7569bf7d" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae"
I20260808 13:10:08.465601 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929f7c72d4b04e019287048d7569bf7d" candidate_uuid: "9b06844055224680b7ca4afa21e813ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d"
I20260808 13:10:08.465772 1727 raft_consensus.cc:3037] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.465833 1891 raft_consensus.cc:3037] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:08.468043 1891 raft_consensus.cc:2454] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b06844055224680b7ca4afa21e813ff in term 1.
I20260808 13:10:08.468395 1502 leader_election.cc:302] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, 9b06844055224680b7ca4afa21e813ff; no voters:
I20260808 13:10:08.468653 2014 raft_consensus.cc:2781] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:08.468689 1727 raft_consensus.cc:2454] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b06844055224680b7ca4afa21e813ff in term 1.
I20260808 13:10:08.468884 2014 raft_consensus.cc:686] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 1 LEADER]: Becoming Leader. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Running, Role: LEADER
I20260808 13:10:08.469305 2014 consensus_queue.cc:237] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:08.472790 1405 catalog_manager.cc:5495] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b06844055224680b7ca4afa21e813ff (127.31.59.129). New cstate: current_term: 1 leader_uuid: "9b06844055224680b7ca4afa21e813ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:08.510877 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.504832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.31.59.129:45777)
0808 13:10:08.505328 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.505332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.505348 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:10:08.505427 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.505431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.505542 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.505676 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.506088 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.506854 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.508701 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.509104 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.509191 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.509704 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.510270 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.510375 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.510521 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.510587 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":271,"threads_started":1}
I20260808 13:10:08.526216 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.520877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.31.59.129:43537)
0808 13:10:08.521363 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:08.521379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:08.521522 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:08.521823 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:08.521827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:08.521839 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:08.522103 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.522112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.523146 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.523151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.524245 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.524257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.525046 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.525053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.525200 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.525252 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.525756 (+ 504us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.525765 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:10:08.525827 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 13:10:08.525935 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":233,"threads_started":1}
I20260808 13:10:08.526271 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.520938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43537 (local address 127.31.59.131:46431)
0808 13:10:08.521136 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.521142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.521476 (+ 334us) server_negotiation.cc:407] Connection header received
0808 13:10:08.521651 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.521655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.521722 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.521806 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.522261 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.523008 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.524383 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.524890 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.525036 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.525505 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.525653 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.525752 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.525862 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.525946 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:10:08.535439 1891 raft_consensus.cc:1261] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Refusing update from remote peer 9b06844055224680b7ca4afa21e813ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:08.535646 1727 raft_consensus.cc:1261] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Refusing update from remote peer 9b06844055224680b7ca4afa21e813ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:08.536208 1972 consensus_queue.cc:1035] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.536808 2014 consensus_queue.cc:1035] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:08.542354 2004 mvcc.cc:205] Tried to move back new op lower bound from 7316253116554186752 to 7316253116294160384. Current Snapshot: MvccSnapshot[applied={T|T < 7316253116554186752}]
I20260808 13:10:08.571583 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.565997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:59411 (local address 127.31.59.129:45777)
0808 13:10:08.566190 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:08.566196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:08.566214 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:10:08.566274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:08.566279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:08.566357 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:08.566445 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:08.567038 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.568140 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.569756 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.570341 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.570476 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.570923 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.571020 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.571119 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:08.571295 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 13:10:08.571369 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 13:10:08.571743 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:08.565386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45777 (local address 127.31.59.131:59411)
0808 13:10:08.565629 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:08.565651 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:08.565776 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:08.566640 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:08.566645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:08.566658 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:08.566862 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.566880 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.568289 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.568293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.569624 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:08.569637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:08.570504 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:08.570511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:08.570716 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:08.570767 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:08.571150 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:08.571164 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 13:10:08.571258 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 13:10:08.571412 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 13:10:08.572902 2004 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 13:10:10.184122 1406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:10.184418 1406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:10.189594 1406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Sending DeleteTablet for 3 replicas of tablet 929f7c72d4b04e019287048d7569bf7d
I20260808 13:10:10.190444 1556 tablet_service.cc:1514] Processing DeleteTablet for tablet 929f7c72d4b04e019287048d7569bf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:49262
I20260808 13:10:10.190681 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 929f7c72d4b04e019287048d7569bf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33510
I20260808 13:10:10.190732 1705 tablet_service.cc:1514] Processing DeleteTablet for tablet 929f7c72d4b04e019287048d7569bf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:32836
I20260808 13:10:10.212935 2067 tablet_replica.cc:330] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: stopping tablet replica
I20260808 13:10:10.213274 2067 raft_consensus.cc:2229] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:10.213667 2067 raft_consensus.cc:2258] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.215029 2067 ts_tablet_manager.cc:1905] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.218114 2066 tablet_replica.cc:330] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: stopping tablet replica
I20260808 13:10:10.218389 2066 raft_consensus.cc:2229] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:10.218585 2065 tablet_replica.cc:330] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: stopping tablet replica
I20260808 13:10:10.218840 2065 raft_consensus.cc:2229] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:10.218918 2066 raft_consensus.cc:2258] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.219550 2065 raft_consensus.cc:2258] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.220032 2066 ts_tablet_manager.cc:1905] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.220598 2065 ts_tablet_manager.cc:1905] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.224747 2067 ts_tablet_manager.cc:1918] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:10.225042 2067 log.cc:1196] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/929f7c72d4b04e019287048d7569bf7d
I20260808 13:10:10.225791 2067 ts_tablet_manager.cc:1939] T 929f7c72d4b04e019287048d7569bf7d P b4306c776c4945459811253878fa8eae: Deleting consensus metadata
I20260808 13:10:10.227000 1389 catalog_manager.cc:4841] TS b4306c776c4945459811253878fa8eae (127.31.59.130:45713): tablet 929f7c72d4b04e019287048d7569bf7d (table test-table [id=3723fe016868440f9217962d7d4161dd]) successfully deleted
I20260808 13:10:10.227974 2066 ts_tablet_manager.cc:1918] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:10.228204 2066 log.cc:1196] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/929f7c72d4b04e019287048d7569bf7d
I20260808 13:10:10.228781 2066 ts_tablet_manager.cc:1939] T 929f7c72d4b04e019287048d7569bf7d P 2434f89e87e646ed89621e3c7721ae3d: Deleting consensus metadata
I20260808 13:10:10.228945 2065 ts_tablet_manager.cc:1918] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:10.229197 2065 log.cc:1196] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/929f7c72d4b04e019287048d7569bf7d
I20260808 13:10:10.229827 2065 ts_tablet_manager.cc:1939] T 929f7c72d4b04e019287048d7569bf7d P 9b06844055224680b7ca4afa21e813ff: Deleting consensus metadata
I20260808 13:10:10.230024 1389 catalog_manager.cc:4841] TS 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431): tablet 929f7c72d4b04e019287048d7569bf7d (table test-table [id=3723fe016868440f9217962d7d4161dd]) successfully deleted
I20260808 13:10:10.230979 1389 catalog_manager.cc:4841] TS 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777): tablet 929f7c72d4b04e019287048d7569bf7d (table test-table [id=3723fe016868440f9217962d7d4161dd]) successfully deleted
Aug 08 13:10:10 dist-test-slave-49h3 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:10:10 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:10.264889 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.247056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.31.59.190:44009)
0808 13:10:10.247988 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.247994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.248210 (+ 216us) server_negotiation.cc:407] Connection header received
0808 13:10:10.248415 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.248419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.248503 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.248584 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:10.250363 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.251295 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.252901 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.253582 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.253693 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.258550 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:10.258556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:10.258559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:10.258593 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:10.262235 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:10.263038 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:10.263041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:10.263044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:10.263186 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:10.263517 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:10.263520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:10.263522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:10.263821 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:10.264015 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.264377 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.264553 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":511,"threads_started":1}
I20260808 13:10:10.268550 1406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:10.269178 1406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:10.275544 1556 tablet_service.cc:1467] Processing CreateTablet for tablet 52c19c2465d6476f9fad2c701be7dfea (DEFAULT_TABLE table=test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.275797 1705 tablet_service.cc:1467] Processing CreateTablet for tablet 52c19c2465d6476f9fad2c701be7dfea (DEFAULT_TABLE table=test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.275985 1556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c19c2465d6476f9fad2c701be7dfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.276291 1705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c19c2465d6476f9fad2c701be7dfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.276216 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 52c19c2465d6476f9fad2c701be7dfea (DEFAULT_TABLE table=test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.276654 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c19c2465d6476f9fad2c701be7dfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.282713 2078 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Bootstrap starting.
I20260808 13:10:10.285519 2078 tablet_bootstrap.cc:654] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.285777 2080 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Bootstrap starting.
I20260808 13:10:10.287024 2079 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Bootstrap starting.
I20260808 13:10:10.288888 2080 tablet_bootstrap.cc:654] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.289752 2079 tablet_bootstrap.cc:654] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.295342 2079 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: No bootstrap required, opened a new log
I20260808 13:10:10.295526 2079 ts_tablet_manager.cc:1397] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260808 13:10:10.296607 2078 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: No bootstrap required, opened a new log
I20260808 13:10:10.296563 2079 raft_consensus.cc:348] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.296839 2079 raft_consensus.cc:374] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.297046 2078 ts_tablet_manager.cc:1397] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.008s
I20260808 13:10:10.297045 2079 raft_consensus.cc:729] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4306c776c4945459811253878fa8eae, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.297405 2079 consensus_queue.cc:260] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.298422 2080 tablet_bootstrap.cc:492] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: No bootstrap required, opened a new log
I20260808 13:10:10.298612 2080 ts_tablet_manager.cc:1397] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260808 13:10:10.298895 2078 raft_consensus.cc:348] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.299335 2078 raft_consensus.cc:374] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.299566 2078 raft_consensus.cc:729] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.299958 2080 raft_consensus.cc:348] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.300335 2080 raft_consensus.cc:374] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.300562 2080 raft_consensus.cc:729] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.300951 2080 consensus_queue.cc:260] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.300001 2078 consensus_queue.cc:260] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.302412 2078 ts_tablet_manager.cc:1428] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260808 13:10:10.304826 2080 ts_tablet_manager.cc:1428] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260808 13:10:10.306007 2079 ts_tablet_manager.cc:1428] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260808 13:10:10.371709 2084 raft_consensus.cc:482] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:10.372140 2084 raft_consensus.cc:504] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.374434 2084 leader_election.cc:288] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:10.383162 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.374793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54141 (local address 127.31.59.129:45777)
0808 13:10:10.375668 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.375675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.375826 (+ 151us) server_negotiation.cc:407] Connection header received
0808 13:10:10.376166 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.376172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.376259 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.376349 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:10.377832 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.379296 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.381120 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.382145 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.382318 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.382388 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.382713 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.382849 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":557,"threads_started":1}
I20260808 13:10:10.383247 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.374733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45777 (local address 127.31.59.130:54141)
0808 13:10:10.375675 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:10.375736 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:10.375899 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:10.376842 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:10.376847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:10.376879 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:10.377594 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.377608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.379444 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.379450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.380959 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.380971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.382332 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.382338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.382522 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.382558 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:10:10.382650 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:10:10.382797 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":1,"spinlock_wait_cycles":3968,"thread_start_us":590,"threads_started":1}
I20260808 13:10:10.383795 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.375759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.31.59.130:48985)
0808 13:10:10.376794 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:10.376815 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:10.376958 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:10.377623 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:10.377626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:10.377650 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:10.378200 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.378215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.379881 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.379884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.381366 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.381380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.382978 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.382984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.383167 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.383203 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:10:10.383308 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 13:10:10.383432 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":1,"thread_start_us":509,"threads_started":1}
I20260808 13:10:10.383800 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52c19c2465d6476f9fad2c701be7dfea" candidate_uuid: "b4306c776c4945459811253878fa8eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff" is_pre_election: true
I20260808 13:10:10.384033 1577 raft_consensus.cc:2454] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4306c776c4945459811253878fa8eae in term 0.
I20260808 13:10:10.384189 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.375916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:48985 (local address 127.31.59.131:46431)
0808 13:10:10.377278 (+ 1362us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.377283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.377321 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:10:10.377392 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.377396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.377495 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.377612 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:10.378478 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.379707 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.381542 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.382609 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.382767 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.382833 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.383593 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.383737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":922,"threads_started":1}
I20260808 13:10:10.384547 1652 leader_election.cc:302] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b06844055224680b7ca4afa21e813ff, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:10.384718 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52c19c2465d6476f9fad2c701be7dfea" candidate_uuid: "b4306c776c4945459811253878fa8eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d" is_pre_election: true
I20260808 13:10:10.384922 2084 raft_consensus.cc:2781] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:10.384971 1891 raft_consensus.cc:2454] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4306c776c4945459811253878fa8eae in term 0.
I20260808 13:10:10.385041 2084 raft_consensus.cc:482] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:10.385144 2084 raft_consensus.cc:3037] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.388531 2084 raft_consensus.cc:504] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.389106 2084 leader_election.cc:288] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [CANDIDATE]: Term 1 election: Requested vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431)
I20260808 13:10:10.389575 1577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52c19c2465d6476f9fad2c701be7dfea" candidate_uuid: "b4306c776c4945459811253878fa8eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff"
I20260808 13:10:10.389626 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52c19c2465d6476f9fad2c701be7dfea" candidate_uuid: "b4306c776c4945459811253878fa8eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434f89e87e646ed89621e3c7721ae3d"
I20260808 13:10:10.389844 1891 raft_consensus.cc:3037] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.389847 1577 raft_consensus.cc:3037] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.393584 1577 raft_consensus.cc:2454] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4306c776c4945459811253878fa8eae in term 1.
I20260808 13:10:10.393622 1891 raft_consensus.cc:2454] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4306c776c4945459811253878fa8eae in term 1.
I20260808 13:10:10.394068 1652 leader_election.cc:302] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b06844055224680b7ca4afa21e813ff, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:10.394474 2084 raft_consensus.cc:2781] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:10.395303 2084 raft_consensus.cc:686] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 1 LEADER]: Becoming Leader. State: Replica: b4306c776c4945459811253878fa8eae, State: Running, Role: LEADER
I20260808 13:10:10.395949 2084 consensus_queue.cc:237] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.400609 1405 catalog_manager.cc:5495] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae reported cstate change: term changed from 0 to 1, leader changed from <none> to b4306c776c4945459811253878fa8eae (127.31.59.130). New cstate: current_term: 1 leader_uuid: "b4306c776c4945459811253878fa8eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:10.428563 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.420618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.31.59.130:45713)
0808 13:10:10.421757 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.421762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.421833 (+ 71us) server_negotiation.cc:407] Connection header received
0808 13:10:10.421912 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.421915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.421987 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.422143 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:10.422689 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.423708 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.425357 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.426197 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.426306 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.427028 (+ 722us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.427731 (+ 703us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.427885 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.428033 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.428111 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":775,"threads_started":1}
I20260808 13:10:10.446906 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.440267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:45657 (local address 127.31.59.131:46431)
0808 13:10:10.440455 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.440459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.441243 (+ 784us) server_negotiation.cc:407] Connection header received
0808 13:10:10.441403 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.441407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.441504 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.441573 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:10.442224 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.443163 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.444919 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.445532 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.445641 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.446358 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.446454 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.446516 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.446610 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.446672 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:10:10.447156 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.440315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46431 (local address 127.31.59.130:45657)
0808 13:10:10.441143 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:10.441159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:10.441288 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:10.441600 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:10.441603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:10.441613 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:10.442101 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.442111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.443275 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.443278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.444793 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.444806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.445793 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.445805 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.446141 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.446215 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.446525 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.446533 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:10:10.446589 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 13:10:10.446687 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":503,"threads_started":1}
I20260808 13:10:10.455821 1577 raft_consensus.cc:1261] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Refusing update from remote peer b4306c776c4945459811253878fa8eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:10.455886 1891 raft_consensus.cc:1261] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Refusing update from remote peer b4306c776c4945459811253878fa8eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:10.456645 2089 consensus_queue.cc:1035] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:10.457226 2084 consensus_queue.cc:1035] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:10.463653 2044 mvcc.cc:205] Tried to move back new op lower bound from 7316253124419002368 to 7316253124187475968. Current Snapshot: MvccSnapshot[applied={T|T < 7316253124419002368}]
I20260808 13:10:10.486539 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.479959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:58393 (local address 127.31.59.130:45713)
0808 13:10:10.480111 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.480116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.480525 (+ 409us) server_negotiation.cc:407] Connection header received
0808 13:10:10.480669 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.480672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.480729 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.480809 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:10.481672 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.482529 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.484522 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.485075 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.485184 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.485798 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.485898 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.485977 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.486152 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.486233 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 13:10:10.486892 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.479613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:45713 (local address 127.31.59.131:58393)
0808 13:10:10.480433 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:10.480448 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:10.480577 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:10.480828 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:10.480832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:10.480964 (+ 132us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:10.481400 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.481411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.482663 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.482668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.484386 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:10.484399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.485315 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.485321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:10.485545 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.485596 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.486022 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.486046 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:10:10.486138 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:10:10.486279 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":509,"threads_started":1}
I20260808 13:10:10.520179 1405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:10.520463 1405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:10.523859 1405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Sending DeleteTablet for 3 replicas of tablet 52c19c2465d6476f9fad2c701be7dfea
I20260808 13:10:10.524785 1556 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c19c2465d6476f9fad2c701be7dfea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:49262
I20260808 13:10:10.525074 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c19c2465d6476f9fad2c701be7dfea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33510
I20260808 13:10:10.525240 1705 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c19c2465d6476f9fad2c701be7dfea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:10 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:32836
I20260808 13:10:10.525424 2065 tablet_replica.cc:330] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: stopping tablet replica
I20260808 13:10:10.525579 2067 tablet_replica.cc:330] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: stopping tablet replica
I20260808 13:10:10.525767 2065 raft_consensus.cc:2229] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:10.525822 2067 raft_consensus.cc:2229] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:10.526055 2065 raft_consensus.cc:2258] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.526441 2066 tablet_replica.cc:330] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: stopping tablet replica
I20260808 13:10:10.526436 2067 raft_consensus.cc:2258] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.526741 2066 raft_consensus.cc:2229] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:10.526966 2066 raft_consensus.cc:2258] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:10.527102 1405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:10.527770 1405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:10.528717 2065 ts_tablet_manager.cc:1905] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.530426 2066 ts_tablet_manager.cc:1905] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.536486 2066 ts_tablet_manager.cc:1918] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:10.536628 2065 ts_tablet_manager.cc:1918] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:10.536681 2066 log.cc:1196] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/52c19c2465d6476f9fad2c701be7dfea
I20260808 13:10:10.536851 2065 log.cc:1196] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/52c19c2465d6476f9fad2c701be7dfea
I20260808 13:10:10.537215 2066 ts_tablet_manager.cc:1939] T 52c19c2465d6476f9fad2c701be7dfea P 2434f89e87e646ed89621e3c7721ae3d: Deleting consensus metadata
I20260808 13:10:10.538560 2067 ts_tablet_manager.cc:1905] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:10.540050 1389 catalog_manager.cc:4841] TS 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431): tablet 52c19c2465d6476f9fad2c701be7dfea (table test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]) successfully deleted
I20260808 13:10:10.541209 2065 ts_tablet_manager.cc:1939] T 52c19c2465d6476f9fad2c701be7dfea P 9b06844055224680b7ca4afa21e813ff: Deleting consensus metadata
I20260808 13:10:10.542745 1556 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e (DEFAULT_TABLE table=test-table [id=d05bf61a8119498ab132714eee83e2e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.543177 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e (DEFAULT_TABLE table=test-table [id=d05bf61a8119498ab132714eee83e2e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.543336 1556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebb6cdef1564d3d88bae5673938f15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.543346 1705 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e (DEFAULT_TABLE table=test-table [id=d05bf61a8119498ab132714eee83e2e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:10.543658 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebb6cdef1564d3d88bae5673938f15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.543833 1705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebb6cdef1564d3d88bae5673938f15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:10.552124 1389 catalog_manager.cc:4841] TS 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777): tablet 52c19c2465d6476f9fad2c701be7dfea (table test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]) successfully deleted
I20260808 13:10:10.553109 2078 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Bootstrap starting.
I20260808 13:10:10.553301 2080 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Bootstrap starting.
I20260808 13:10:10.555752 2078 tablet_bootstrap.cc:654] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.556438 2080 tablet_bootstrap.cc:654] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.557536 2067 ts_tablet_manager.cc:1918] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:10.559615 2079 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Bootstrap starting.
I20260808 13:10:10.559772 2067 log.cc:1196] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/52c19c2465d6476f9fad2c701be7dfea
I20260808 13:10:10.560350 2067 ts_tablet_manager.cc:1939] T 52c19c2465d6476f9fad2c701be7dfea P b4306c776c4945459811253878fa8eae: Deleting consensus metadata
I20260808 13:10:10.562606 1389 catalog_manager.cc:4841] TS b4306c776c4945459811253878fa8eae (127.31.59.130:45713): tablet 52c19c2465d6476f9fad2c701be7dfea (table test-table [id=7b376a5eb91344d2bcd383ecfcd32e50]) successfully deleted
I20260808 13:10:10.563083 2079 tablet_bootstrap.cc:654] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:10.564244 2078 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: No bootstrap required, opened a new log
I20260808 13:10:10.564488 2078 ts_tablet_manager.cc:1397] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260808 13:10:10.565701 2078 raft_consensus.cc:348] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.566015 2078 raft_consensus.cc:374] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.566176 2078 raft_consensus.cc:729] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b06844055224680b7ca4afa21e813ff, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.566552 2078 consensus_queue.cc:260] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.567301 2078 ts_tablet_manager.cc:1428] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 13:10:10.571810 2080 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: No bootstrap required, opened a new log
I20260808 13:10:10.572118 2080 ts_tablet_manager.cc:1397] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.000s
I20260808 13:10:10.572392 2079 tablet_bootstrap.cc:492] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: No bootstrap required, opened a new log
I20260808 13:10:10.572631 2079 ts_tablet_manager.cc:1397] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260808 13:10:10.573374 2080 raft_consensus.cc:348] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.573696 2080 raft_consensus.cc:374] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.573818 2080 raft_consensus.cc:729] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.573766 2079 raft_consensus.cc:348] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.574126 2080 consensus_queue.cc:260] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.574244 2079 raft_consensus.cc:374] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:10.574381 2079 raft_consensus.cc:729] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4306c776c4945459811253878fa8eae, State: Initialized, Role: FOLLOWER
I20260808 13:10:10.574820 2079 consensus_queue.cc:260] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.576152 2079 ts_tablet_manager.cc:1428] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:10.576603 2080 ts_tablet_manager.cc:1428] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 13:10:10.716815 2102 raft_consensus.cc:482] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:10.717098 2102 raft_consensus.cc:504] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.718371 2102 leader_election.cc:288] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:10.719559 1728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebb6cdef1564d3d88bae5673938f15e" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae" is_pre_election: true
I20260808 13:10:10.719627 1575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebb6cdef1564d3d88bae5673938f15e" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff" is_pre_election: true
I20260808 13:10:10.719900 1575 raft_consensus.cc:2454] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:10.719990 1728 raft_consensus.cc:2454] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 0.
I20260808 13:10:10.720620 1808 leader_election.cc:302] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, b4306c776c4945459811253878fa8eae; no voters:
I20260808 13:10:10.720966 2102 raft_consensus.cc:2781] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:10.721087 2102 raft_consensus.cc:482] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:10.721153 2102 raft_consensus.cc:3037] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.725029 2102 raft_consensus.cc:504] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.725800 2102 leader_election.cc:288] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Requested vote from peers 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777), b4306c776c4945459811253878fa8eae (127.31.59.130:45713)
I20260808 13:10:10.726544 1575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebb6cdef1564d3d88bae5673938f15e" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b06844055224680b7ca4afa21e813ff"
I20260808 13:10:10.726894 1575 raft_consensus.cc:3037] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.729053 1728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebb6cdef1564d3d88bae5673938f15e" candidate_uuid: "2434f89e87e646ed89621e3c7721ae3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4306c776c4945459811253878fa8eae"
I20260808 13:10:10.729434 1728 raft_consensus.cc:3037] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:10.732376 1575 raft_consensus.cc:2454] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 1.
I20260808 13:10:10.733069 1808 leader_election.cc:302] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434f89e87e646ed89621e3c7721ae3d, 9b06844055224680b7ca4afa21e813ff; no voters:
I20260808 13:10:10.733266 1728 raft_consensus.cc:2454] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2434f89e87e646ed89621e3c7721ae3d in term 1.
I20260808 13:10:10.734078 2102 raft_consensus.cc:2781] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:10.734320 2102 raft_consensus.cc:686] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 1 LEADER]: Becoming Leader. State: Replica: 2434f89e87e646ed89621e3c7721ae3d, State: Running, Role: LEADER
I20260808 13:10:10.734771 2102 consensus_queue.cc:237] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } }
I20260808 13:10:10.738137 1406 catalog_manager.cc:5495] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131). New cstate: current_term: 1 leader_uuid: "2434f89e87e646ed89621e3c7721ae3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2434f89e87e646ed89621e3c7721ae3d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46431 } health_report { overall_health: HEALTHY } } }
I20260808 13:10:10.763258 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:10.756855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.31.59.131:46431)
0808 13:10:10.757192 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:10.757198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:10.757216 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:10:10.757279 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:10.757284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:10.757381 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:10.757527 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:10.758268 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.759413 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.761050 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:10.761862 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:10.761976 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:10.762458 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:10.762559 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:10.762633 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:10.762803 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:10:10.762886 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260808 13:10:10.775213 1728 raft_consensus.cc:1261] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Refusing update from remote peer 2434f89e87e646ed89621e3c7721ae3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:10.775322 1575 raft_consensus.cc:1261] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Refusing update from remote peer 2434f89e87e646ed89621e3c7721ae3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:10.775995 2043 consensus_queue.cc:1035] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b06844055224680b7ca4afa21e813ff" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:10.776667 2102 consensus_queue.cc:1035] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4306c776c4945459811253878fa8eae" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:10:10.804994 2041 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 13:10:12.405514 1406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:12.405781 1406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:12.411453 1406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7c9a91983c41b89de96652a08145e0: Sending DeleteTablet for 3 replicas of tablet 5ebb6cdef1564d3d88bae5673938f15e
I20260808 13:10:12.412367 1556 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:12 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:49262
I20260808 13:10:12.412390 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:12 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33510
I20260808 13:10:12.412457 1705 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebb6cdef1564d3d88bae5673938f15e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:12 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:32836
I20260808 13:10:12.414180 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1485
I20260808 13:10:12.425246 2136 tablet_replica.cc:330] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: stopping tablet replica
I20260808 13:10:12.425539 2136 raft_consensus.cc:2229] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:12.425896 2136 raft_consensus.cc:2258] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:12.427151 2136 ts_tablet_manager.cc:1905] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:12.428052 2137 tablet_replica.cc:330] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: stopping tablet replica
I20260808 13:10:12.428324 2137 raft_consensus.cc:2229] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:12.428934 2139 tablet_replica.cc:330] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: stopping tablet replica
I20260808 13:10:12.428921 2137 raft_consensus.cc:2258] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:12.429198 2139 raft_consensus.cc:2229] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:12.429680 2139 raft_consensus.cc:2258] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:12.430560 2137 ts_tablet_manager.cc:1905] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:12.435545 2136 ts_tablet_manager.cc:1918] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:12.435753 2136 log.cc:1196] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/5ebb6cdef1564d3d88bae5673938f15e
I20260808 13:10:12.436367 2136 ts_tablet_manager.cc:1939] T 5ebb6cdef1564d3d88bae5673938f15e P 9b06844055224680b7ca4afa21e813ff: Deleting consensus metadata
I20260808 13:10:12.437620 1389 catalog_manager.cc:4841] TS 9b06844055224680b7ca4afa21e813ff (127.31.59.129:45777): tablet 5ebb6cdef1564d3d88bae5673938f15e (table test-table [id=d05bf61a8119498ab132714eee83e2e9]) successfully deleted
I20260808 13:10:12.438025 2137 ts_tablet_manager.cc:1918] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:12.438050 2139 ts_tablet_manager.cc:1905] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:12.438274 2137 log.cc:1196] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/5ebb6cdef1564d3d88bae5673938f15e
I20260808 13:10:12.438906 2137 ts_tablet_manager.cc:1939] T 5ebb6cdef1564d3d88bae5673938f15e P 2434f89e87e646ed89621e3c7721ae3d: Deleting consensus metadata
I20260808 13:10:12.440169 1389 catalog_manager.cc:4841] TS 2434f89e87e646ed89621e3c7721ae3d (127.31.59.131:46431): tablet 5ebb6cdef1564d3d88bae5673938f15e (table test-table [id=d05bf61a8119498ab132714eee83e2e9]) successfully deleted
I20260808 13:10:12.444674 2139 ts_tablet_manager.cc:1918] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:10:12.444947 2139 log.cc:1196] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/5ebb6cdef1564d3d88bae5673938f15e
I20260808 13:10:12.445567 2139 ts_tablet_manager.cc:1939] T 5ebb6cdef1564d3d88bae5673938f15e P b4306c776c4945459811253878fa8eae: Deleting consensus metadata
I20260808 13:10:12.446573 1389 catalog_manager.cc:4841] TS b4306c776c4945459811253878fa8eae (127.31.59.130:45713): tablet 5ebb6cdef1564d3d88bae5673938f15e (table test-table [id=d05bf61a8119498ab132714eee83e2e9]) successfully deleted
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:12.455690 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:12.416444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.31.59.129:45777)
0808 13:10:12.430706 (+ 14262us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:12.430712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:12.430747 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:10:12.430807 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:12.430812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:12.430884 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:12.430986 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:12.433413 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.435084 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.437198 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.438715 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.438845 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:12.448682 (+ 9837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:12.448701 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:12.448717 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:12.448829 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:12.452761 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.453434 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.453441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.453465 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.453724 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.454130 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.454135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.454139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.454671 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:12.454985 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:12.455136 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 13:10:12.455328 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14124,"thread_start_us":13414,"threads_started":1}
W20260808 13:10:12.456579 1625 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49278
W20260808 13:10:12.457281 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1485: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:12.457679 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 1485
I20260808 13:10:12.482110 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1634
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:12.496894 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:12.482904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.31.59.130:45713)
0808 13:10:12.483468 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:12.483472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:12.483488 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:10:12.483543 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:12.483547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:12.483604 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:12.483692 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:12.485217 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.486061 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.487338 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.487733 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.487824 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:12.491840 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:12.491855 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:12.491871 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:12.491911 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:12.494632 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.495192 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.495198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.495200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.495345 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.495613 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.495616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.495618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.495961 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:12.496188 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:12.496384 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 13:10:12.496551 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":316,"threads_started":1}
W20260808 13:10:12.497704 1778 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32850
W20260808 13:10:12.498333 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:12.498468 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 1634
I20260808 13:10:12.526269 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1788
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:12.540831 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:12.527008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.31.59.131:46431)
0808 13:10:12.527520 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:12.527525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:12.527542 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:10:12.527598 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:12.527602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:12.527680 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:12.527769 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:12.529098 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.529909 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.531074 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:12.531479 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:12.531571 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:12.535647 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:12.535664 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:12.535680 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:12.535713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:12.538579 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.539189 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.539196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.539199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.539331 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:12.539602 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:12.539605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:12.539607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:12.539971 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:12.540204 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:12.540395 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 13:10:12.540558 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":250,"threads_started":1}
W20260808 13:10:12.541539 1937 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33548
W20260808 13:10:12.542120 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1788: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:12.542284 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 1788
I20260808 13:10:12.567983 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 1369
I20260808 13:10:12.569156 1451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:12.686558 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 1369
2026-08-08T13:10:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8781 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[2148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[2148](info): set up 2 sockets
Aug 08 13:10:12 dist-test-slave-49h3 krb5kdc[2148](info): commencing operation
krb5kdc: starting...
W20260808 13:10:13.741293 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:13 dist-test-slave-49h3 krb5kdc[2148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:13.940659 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38993
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:14.271713 2162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:14.271983 2162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:14.272117 2162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:14.279466 2162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:14.279577 2162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:14.279659 2162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:14.279752 2162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:14.279803 2162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:14.289659 2162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:14.290946 2162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:14.292335 2162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:14.304196 2167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:14.304242 2171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:14.304360 2162 server_base.cc:1034] running on GCE node
W20260808 13:10:14.304150 2168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:14.305477 2162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:14.306767 2162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:14.307960 2162 hybrid_clock.cc:648] HybridClock initialized: now 1786194614307911 us; error 31 us; skew 500 ppm
I20260808 13:10:14.308285 2162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:14 dist-test-slave-49h3 krb5kdc[2148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194614, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:14.311328 2162 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:14.312757 2162 webserver.cc:460] Webserver started at http://127.31.59.190:43779/ using document root <none> and password file <none>
I20260808 13:10:14.313524 2162 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:14.313642 2162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:14.313951 2162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:14.316924 2162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "2a1449adb20b44a28a3afe7bbee9bfc7"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "45bf345e0778437408776bd24e3d5dbd"
server_key_iv: "fb89535f414ea6a37d70b56493f4c1bc"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.317718 2162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "2a1449adb20b44a28a3afe7bbee9bfc7"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "45bf345e0778437408776bd24e3d5dbd"
server_key_iv: "fb89535f414ea6a37d70b56493f4c1bc"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.322716 2162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:10:14.326182 2178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:14.327407 2162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:14.327561 2162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2a1449adb20b44a28a3afe7bbee9bfc7"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "45bf345e0778437408776bd24e3d5dbd"
server_key_iv: "fb89535f414ea6a37d70b56493f4c1bc"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.327737 2162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:14.339478 2162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:14.343097 2162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:14.343369 2162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:14.360734 2162 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:38993
I20260808 13:10:14.360741 2230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:38993 every 8 connection(s)
I20260808 13:10:14.362120 2162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:14.364328 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2162
I20260808 13:10:14.364537 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:14.365186 31982 external_mini_cluster.cc:1428] Setting key 6f951e742d52695e225d41f864177797
I20260808 13:10:14.365773 2231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:14.379699 2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7: Bootstrap starting.
Aug 08 13:10:14 dist-test-slave-49h3 krb5kdc[2148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:14.382866 2231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:14.383899 2231 log.cc:824] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:14.386658 2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7: No bootstrap required, opened a new log
I20260808 13:10:14.387961 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:14.367062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.31.59.190:38993)
0808 13:10:14.367804 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:14.367815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:14.367854 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:10:14.370177 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:14.370200 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:14.370580 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:14.370894 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:14.372666 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.373982 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.375720 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.376431 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.376559 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:14.381053 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:14.381062 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:14.381079 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:14.381119 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:14.384130 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:14.384769 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:14.384776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:14.384782 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:14.384956 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:14.385315 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:14.385318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:14.385321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:14.385866 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:14.386151 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:14.386587 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 13:10:14.387020 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":332,"threads_started":1}
I20260808 13:10:14.394021 2231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } }
I20260808 13:10:14.394378 2231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:14.394474 2231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1449adb20b44a28a3afe7bbee9bfc7, State: Initialized, Role: FOLLOWER
I20260808 13:10:14.395012 2231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } }
I20260808 13:10:14.395236 2231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:14.395393 2231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:14.395587 2231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:14.397871 2231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } }
I20260808 13:10:14.398361 2231 leader_election.cc:302] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1449adb20b44a28a3afe7bbee9bfc7; no voters:
I20260808 13:10:14.398914 2231 leader_election.cc:288] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:14.399040 2236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:14.400146 2236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1449adb20b44a28a3afe7bbee9bfc7, State: Running, Role: LEADER
I20260808 13:10:14.400627 2236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } }
I20260808 13:10:14.400980 2231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:14.403362 2237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } } }
I20260808 13:10:14.403348 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a1449adb20b44a28a3afe7bbee9bfc7. Latest consensus state: current_term: 1 leader_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1449adb20b44a28a3afe7bbee9bfc7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38993 } } }
I20260808 13:10:14.403798 2237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:14.403798 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:14.405344 2242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:14.410133 2242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:14.420049 2242 catalog_manager.cc:1347] Generated new cluster ID: 6a635a374bc446ec96a503b1560110df
I20260808 13:10:14.420150 2242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:14.440825 2242 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:14.442207 2242 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:14.450302 2242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a1449adb20b44a28a3afe7bbee9bfc7: Generated new TSK 0
I20260808 13:10:14.451121 2242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:14.502789 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:14.831071 2257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:14.831388 2257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:14.831537 2257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:14.838714 2257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:14.838826 2257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:14.838977 2257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:14.848600 2257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:14.849942 2257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:14.851269 2257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:14.863776 2263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:14.863848 2265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:14.863875 2262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:14.864629 2257 server_base.cc:1034] running on GCE node
I20260808 13:10:14.865214 2257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:14.866191 2257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:14.867388 2257 hybrid_clock.cc:648] HybridClock initialized: now 1786194614867338 us; error 40 us; skew 500 ppm
I20260808 13:10:14.867683 2257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:14 dist-test-slave-49h3 krb5kdc[2148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194614, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:14.870759 2257 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:14.872174 2257 webserver.cc:460] Webserver started at http://127.31.59.129:36591/ using document root <none> and password file <none>
I20260808 13:10:14.872831 2257 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:14.872947 2257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:14.873248 2257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:14.876143 2257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "26c6bd6066064433b02f2736c441582d"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "eaa40fc6cf04a0ae83d13aac7df96590"
server_key_iv: "bf07ed2a38404bef36a9a1738cac2905"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.876916 2257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "26c6bd6066064433b02f2736c441582d"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "eaa40fc6cf04a0ae83d13aac7df96590"
server_key_iv: "bf07ed2a38404bef36a9a1738cac2905"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.881937 2257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260808 13:10:14.885179 2273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:14.886454 2257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 13:10:14.886600 2257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "26c6bd6066064433b02f2736c441582d"
format_stamp: "Formatted at 2026-08-08 13:10:14 on dist-test-slave-49h3"
server_key: "eaa40fc6cf04a0ae83d13aac7df96590"
server_key_iv: "bf07ed2a38404bef36a9a1738cac2905"
server_key_version: "encryptionkey@0"
I20260808 13:10:14.886786 2257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:14.904893 2257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:14.908743 2257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:14.909020 2257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:14.909997 2257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:14.911314 2257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:14.911393 2257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:14.911516 2257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:14.911571 2257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:14.943250 2257 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:33427
I20260808 13:10:14.943307 2391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:33427 every 8 connection(s)
I20260808 13:10:14.944482 2257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:14.953285 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2257
I20260808 13:10:14.953508 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:14.954107 31982 external_mini_cluster.cc:1428] Setting key c08e25ece52e8a84a9fb108657d34fba
Aug 08 13:10:14 dist-test-slave-49h3 krb5kdc[2148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194614, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:14.964182 2394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:14.946882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:51875 (local address 127.31.59.190:38993)
0808 13:10:14.947308 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:14.947312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:14.948260 (+ 948us) server_negotiation.cc:407] Connection header received
0808 13:10:14.950132 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:14.950138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:14.950215 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:14.950335 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:14.951768 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.952617 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.954287 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.954690 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.954788 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:14.958330 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:14.958336 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:14.958341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:14.958378 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:14.961243 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:14.961972 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:14.961977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:14.961980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:14.962124 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:14.962482 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:14.962486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:14.962488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:14.962817 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:14.962944 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:14.963606 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 13:10:14.963831 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260808 13:10:14.965338 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:14.947416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38993 (local address 127.31.59.129:51875)
0808 13:10:14.948110 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:14.948148 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:14.949875 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:14.950553 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:14.950561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:14.951065 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:14.951602 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:14.951618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.952767 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.952772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:14.954112 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:14.954124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:14.954906 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:14.954912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:14.955160 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:14.955889 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:14.955926 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:14.958086 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:14.961370 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:14.961377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:14.961389 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:14.961846 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:14.962208 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:14.962211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:14.962213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:14.962386 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:14.962922 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:14.962927 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:14.963232 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 13:10:14.963463 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 13:10:14.963990 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":283,"threads_started":1}
I20260808 13:10:14.966823 2392 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38993
I20260808 13:10:14.967133 2392 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:14.968150 2392 heartbeater.cc:511] Master 127.31.59.190:38993 requested a full tablet report, sending...
I20260808 13:10:14.970273 2195 ts_manager.cc:194] Registered new tserver with Master: 26c6bd6066064433b02f2736c441582d (127.31.59.129:33427)
I20260808 13:10:14.971676 2195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:51875
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:15.000547 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:15.377516 2398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:15.377796 2398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:15.377903 2398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:15.385167 2398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:15.385277 2398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:15.385398 2398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:15.395136 2398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:15.396411 2398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:15.397747 2398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:15.414355 2405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:15.415047 2407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:15.415407 2404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:15.416047 2398 server_base.cc:1034] running on GCE node
I20260808 13:10:15.416625 2398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:15.417570 2398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:15.418776 2398 hybrid_clock.cc:648] HybridClock initialized: now 1786194615418722 us; error 42 us; skew 500 ppm
I20260808 13:10:15.419080 2398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:15 dist-test-slave-49h3 krb5kdc[2148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194615, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:15.422116 2398 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:15.423566 2398 webserver.cc:460] Webserver started at http://127.31.59.130:45091/ using document root <none> and password file <none>
I20260808 13:10:15.424229 2398 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:15.424351 2398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:15.424640 2398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:15.427569 2398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "481ce3abf6a04dfc86b3fcbbdc4ec5c6"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "4b735982eb0682362b3e1a9f0a8a759f"
server_key_iv: "fb9045f691e6eeeddf0c2eba39ae6a71"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.428434 2398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "481ce3abf6a04dfc86b3fcbbdc4ec5c6"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "4b735982eb0682362b3e1a9f0a8a759f"
server_key_iv: "fb9045f691e6eeeddf0c2eba39ae6a71"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.433677 2398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:10:15.436928 2414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.438215 2398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:15.438374 2398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "481ce3abf6a04dfc86b3fcbbdc4ec5c6"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "4b735982eb0682362b3e1a9f0a8a759f"
server_key_iv: "fb9045f691e6eeeddf0c2eba39ae6a71"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.438546 2398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:15.452137 2398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:15.455796 2398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:15.456058 2398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:15.456964 2398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:15.458297 2398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:15.458457 2398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.458580 2398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:15.458638 2398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.488046 2398 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44863
I20260808 13:10:15.488094 2530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44863 every 8 connection(s)
I20260808 13:10:15.489300 2398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:15.492808 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2398
I20260808 13:10:15.493016 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:15.493639 31982 external_mini_cluster.cc:1428] Setting key 615973a8c12ca81c011430b520a05fb5
Aug 08 13:10:15 dist-test-slave-49h3 krb5kdc[2148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194615, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:15.510679 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:15.491866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:47265 (local address 127.31.59.190:38993)
0808 13:10:15.492330 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:15.492336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:15.493490 (+ 1154us) server_negotiation.cc:407] Connection header received
0808 13:10:15.495478 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:15.495483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:15.495565 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:15.495669 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:15.497726 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.498454 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.500509 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.500963 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.501070 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:15.505207 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:15.505212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:15.505215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:15.505247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:15.507755 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:15.508435 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:15.508439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:15.508441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:15.508562 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:15.508973 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:15.508976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:15.508978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:15.509239 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:15.509371 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:15.510087 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 13:10:15.510321 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":216,"threads_started":1}
I20260808 13:10:15.511628 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:15.492379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38993 (local address 127.31.59.130:47265)
0808 13:10:15.493300 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:15.493351 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:15.495235 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:15.495946 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:15.495958 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:15.496737 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:15.497552 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:15.497578 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.498611 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.498617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:15.500361 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:15.500376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.501202 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.501209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:15.501514 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:15.502276 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:15.502304 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:15.504997 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:15.507890 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:15.507898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:15.507914 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:15.508317 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:15.508665 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:15.508670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:15.508674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:15.508863 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:15.509364 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:15.509371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:15.509736 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 13:10:15.509969 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 13:10:15.510490 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":316,"threads_started":1}
I20260808 13:10:15.512995 2531 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38993
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
I20260808 13:10:15.513339 2531 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:15.514210 2531 heartbeater.cc:511] Master 127.31.59.190:38993 requested a full tablet report, sending...
I20260808 13:10:15.515643 2195 ts_manager.cc:194] Registered new tserver with Master: 481ce3abf6a04dfc86b3fcbbdc4ec5c6 (127.31.59.130:44863)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
I20260808 13:10:15.516520 2195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:47265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:15.538250 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:15.855247 2537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:15.855520 2537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:15.855623 2537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:15.863149 2537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:15.863272 2537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:15.863399 2537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:15.873273 2537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:15.874614 2537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:15.875888 2537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:15.888916 2543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:15.888788 2546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:15.888797 2544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:15.889513 2537 server_base.cc:1034] running on GCE node
I20260808 13:10:15.890215 2537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:15.891209 2537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:15.892395 2537 hybrid_clock.cc:648] HybridClock initialized: now 1786194615892343 us; error 50 us; skew 500 ppm
I20260808 13:10:15.892706 2537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:15 dist-test-slave-49h3 krb5kdc[2148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194615, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:15.895788 2537 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:15.897185 2537 webserver.cc:460] Webserver started at http://127.31.59.131:36969/ using document root <none> and password file <none>
I20260808 13:10:15.897912 2537 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:15.898025 2537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:15.898332 2537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:15.901279 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "a224180a9bc6437b882ee9c96c8bcd23"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "5d065dccd18bb90da4136d79bb11aefe"
server_key_iv: "d335a65b0bb75b8413a680c777ef539b"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.902118 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "a224180a9bc6437b882ee9c96c8bcd23"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "5d065dccd18bb90da4136d79bb11aefe"
server_key_iv: "d335a65b0bb75b8413a680c777ef539b"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.907397 2537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:10:15.910849 2553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.912191 2537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:15.912350 2537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "a224180a9bc6437b882ee9c96c8bcd23"
format_stamp: "Formatted at 2026-08-08 13:10:15 on dist-test-slave-49h3"
server_key: "5d065dccd18bb90da4136d79bb11aefe"
server_key_iv: "d335a65b0bb75b8413a680c777ef539b"
server_key_version: "encryptionkey@0"
I20260808 13:10:15.912536 2537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:15.933619 2537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:15.939443 2537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:15.939778 2537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:15.940994 2537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:15.942664 2537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:15.942749 2537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.942919 2537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:15.942970 2537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:15.974367 2537 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:35833
I20260808 13:10:15.974452 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:35833 every 8 connection(s)
I20260808 13:10:15.974805 2392 heartbeater.cc:503] Master 127.31.59.190:38993 was elected leader, sending a full tablet report...
I20260808 13:10:15.975819 2537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:15.984778 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2537
I20260808 13:10:15.984964 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:15.985373 31982 external_mini_cluster.cc:1428] Setting key 772c77e6fba193278e394753913b84d4
Aug 08 13:10:15 dist-test-slave-49h3 krb5kdc[2148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194615, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:15.995024 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:15.978326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:59261 (local address 127.31.59.190:38993)
0808 13:10:15.978516 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:15.978522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:15.979786 (+ 1264us) server_negotiation.cc:407] Connection header received
0808 13:10:15.981579 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:15.981584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:15.981674 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:15.981758 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:15.983194 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.983913 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.985633 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.986142 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.986241 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:15.989827 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:15.989832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:15.989835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:15.989867 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:15.992397 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:15.993078 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:15.993082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:15.993084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:15.993205 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:15.993578 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:15.993581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:15.993582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:15.993868 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:15.993985 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:15.994608 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 13:10:15.994798 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:10:15.996017 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:15.978902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38993 (local address 127.31.59.131:59261)
0808 13:10:15.979620 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:15.979665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:15.981302 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:15.981949 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:15.981957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:15.982511 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:15.983063 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:15.983079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.984089 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.984094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:15.985439 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:15.985493 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:15.986376 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:15.986384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:15.986617 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:15.987346 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:15.987379 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:15.989620 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:15.992523 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:15.992530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:15.992545 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:15.992975 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:15.993282 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:15.993285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:15.993287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:15.993500 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:15.993958 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:15.993963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:15.994297 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 13:10:15.994522 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 13:10:15.994911 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260808 13:10:15.997402 2672 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38993
I20260808 13:10:15.997720 2672 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:15.998427 2672 heartbeater.cc:511] Master 127.31.59.190:38993 requested a full tablet report, sending...
I20260808 13:10:15.999892 2195 ts_manager.cc:194] Registered new tserver with Master: a224180a9bc6437b882ee9c96c8bcd23 (127.31.59.131:35833)
I20260808 13:10:16.000787 2195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:59261
I20260808 13:10:16.001008 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:10:16.072938 2533 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:16.070455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.31.59.190:38993)
0808 13:10:16.070628 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:16.070632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:16.071087 (+ 455us) server_negotiation.cc:407] Connection header received
0808 13:10:16.071240 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:16.071244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:16.071324 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:16.071416 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:16.072664 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37030: BlockingRecv error: recv got EOF from 127.0.0.1:37030 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:10:16.076361 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2257
W20260808 13:10:16.079077 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2257: Not authorized: Client connection negotiation failed: client connection to 127.31.59.129:33427: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:10:16.079259 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2257
I20260808 13:10:16.097771 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2398
W20260808 13:10:16.100418 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2398: Not authorized: Client connection negotiation failed: client connection to 127.31.59.130:44863: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:10:16.100571 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2398
I20260808 13:10:16.117357 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2537
W20260808 13:10:16.120386 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2537: Not authorized: Client connection negotiation failed: client connection to 127.31.59.131:35833: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:10:16.120522 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2537
I20260808 13:10:16.136296 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2162
I20260808 13:10:16.137677 2226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:16.244199 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2162
2026-08-08T13:10:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3555 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:16 dist-test-slave-49h3 krb5kdc[2689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:16 dist-test-slave-49h3 krb5kdc[2689](info): set up 2 sockets
Aug 08 13:10:16 dist-test-slave-49h3 krb5kdc[2689](info): commencing operation
krb5kdc: starting...
W20260808 13:10:17.288123 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:17 dist-test-slave-49h3 krb5kdc[2689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:17.492175 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37863
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:17.819103 2702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:17.819412 2702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:17.819557 2702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:17.826802 2702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:17.826926 2702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:17.827020 2702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:17.827096 2702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:17.827163 2702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:17.837208 2702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:37863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:17.838564 2702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:17.839857 2702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:17.851500 2708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:17.851542 2709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:17.851457 2711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:17.852420 2702 server_base.cc:1034] running on GCE node
I20260808 13:10:17.853163 2702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:17.854624 2702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:17.855818 2702 hybrid_clock.cc:648] HybridClock initialized: now 1786194617855770 us; error 38 us; skew 500 ppm
I20260808 13:10:17.856156 2702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:17 dist-test-slave-49h3 krb5kdc[2689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194617, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:17.862322 2702 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:17.863902 2702 webserver.cc:460] Webserver started at http://127.31.59.190:41839/ using document root <none> and password file <none>
I20260808 13:10:17.864610 2702 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:17.864732 2702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:17.865033 2702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:17.868198 2702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "419b75ee6b674f01ba756e65b8ce94c2"
format_stamp: "Formatted at 2026-08-08 13:10:17 on dist-test-slave-49h3"
server_key: "8918610c2fe536b884cacfc8c484ee7f"
server_key_iv: "e98a3c7f1bfa3a6b6a9cf0d162350ca4"
server_key_version: "encryptionkey@0"
I20260808 13:10:17.869006 2702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "419b75ee6b674f01ba756e65b8ce94c2"
format_stamp: "Formatted at 2026-08-08 13:10:17 on dist-test-slave-49h3"
server_key: "8918610c2fe536b884cacfc8c484ee7f"
server_key_iv: "e98a3c7f1bfa3a6b6a9cf0d162350ca4"
server_key_version: "encryptionkey@0"
I20260808 13:10:17.874344 2702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:10:17.877766 2719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:17.879106 2702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:17.879281 2702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "419b75ee6b674f01ba756e65b8ce94c2"
format_stamp: "Formatted at 2026-08-08 13:10:17 on dist-test-slave-49h3"
server_key: "8918610c2fe536b884cacfc8c484ee7f"
server_key_iv: "e98a3c7f1bfa3a6b6a9cf0d162350ca4"
server_key_version: "encryptionkey@0"
I20260808 13:10:17.879479 2702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:17.896178 2702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:17.901016 2702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:17.901360 2702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:17.921726 2702 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:37863
I20260808 13:10:17.921759 2776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:37863 every 8 connection(s)
I20260808 13:10:17.923205 2702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:17.927132 2777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:17.932174 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2702
I20260808 13:10:17.932426 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:17.933277 31982 external_mini_cluster.cc:1428] Setting key a3324b2605cf1c92aee0e5e2eeaec455
I20260808 13:10:17.939065 2777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2: Bootstrap starting.
I20260808 13:10:17.943200 2777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:17.944617 2777 log.cc:824] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:17.947913 2777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2: No bootstrap required, opened a new log
Aug 08 13:10:17 dist-test-slave-49h3 krb5kdc[2689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:17.956200 2777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } }
I20260808 13:10:17.956740 2777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:17.956874 2777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419b75ee6b674f01ba756e65b8ce94c2, State: Initialized, Role: FOLLOWER
I20260808 13:10:17.957620 2777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } }
I20260808 13:10:17.957907 2777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:17.958034 2777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:17.958202 2777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:17.962121 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:17.935538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.31.59.190:37863)
0808 13:10:17.936880 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:17.936905 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:17.936964 (+ 59us) server_negotiation.cc:407] Connection header received
0808 13:10:17.939657 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:17.939690 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:17.940185 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:17.940634 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:17.943498 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:17.945502 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:17.947238 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:17.948046 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:17.948235 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:17.952926 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:17.952936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:17.952973 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:17.953017 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:17.957411 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:17.958359 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:17.958366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:17.958373 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:17.958564 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:17.958965 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:17.958970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:17.958974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:17.959629 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:17.959934 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:17.960483 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 13:10:17.960930 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":550,"threads_started":1}
I20260808 13:10:17.966082 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:10:17.982821 2777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } }
I20260808 13:10:17.983631 2777 leader_election.cc:302] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 419b75ee6b674f01ba756e65b8ce94c2; no voters:
I20260808 13:10:17.985802 2777 leader_election.cc:288] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:17.986006 2783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:17.988103 2777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:17.990128 2783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [term 1 LEADER]: Becoming Leader. State: Replica: 419b75ee6b674f01ba756e65b8ce94c2, State: Running, Role: LEADER
I20260808 13:10:17.990801 2783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } }
I20260808 13:10:17.993815 2786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 419b75ee6b674f01ba756e65b8ce94c2. Latest consensus state: current_term: 1 leader_uuid: "419b75ee6b674f01ba756e65b8ce94c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } } }
I20260808 13:10:17.994212 2786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:17.994699 2784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "419b75ee6b674f01ba756e65b8ce94c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b75ee6b674f01ba756e65b8ce94c2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37863 } } }
I20260808 13:10:17.994982 2784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:17.996845 2795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:18.002203 2795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:18.010905 2795 catalog_manager.cc:1347] Generated new cluster ID: 93bd3b354adc408fa0a043617303081f
I20260808 13:10:18.011020 2795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:18.023970 2795 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:18.025568 2795 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:18.028270 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:18.018229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.31.59.190:37863)
0808 13:10:18.018419 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:18.018425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:18.019075 (+ 650us) server_negotiation.cc:407] Connection header received
0808 13:10:18.019264 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:18.019269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:18.019347 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:18.019447 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:18.020826 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:18.021937 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:18.024010 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:18.024656 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:18.024801 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:18.026358 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:18.026370 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:18.026376 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:18.027349 (+ 973us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:18.027425 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:18.027457 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:18.027567 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:18.027743 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 13:10:18.027884 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":66}
I20260808 13:10:18.037005 2795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 419b75ee6b674f01ba756e65b8ce94c2: Generated new TSK 0
I20260808 13:10:18.038036 2795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:10:18.049275 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2702
I20260808 13:10:18.051141 2771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:18.142377 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2702
2026-08-08T13:10:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1892 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:19 dist-test-slave-49h3 krb5kdc[2813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:19 dist-test-slave-49h3 krb5kdc[2813](info): set up 2 sockets
Aug 08 13:10:19 dist-test-slave-49h3 krb5kdc[2813](info): commencing operation
krb5kdc: starting...
W20260808 13:10:20.753216 31982 mini_kdc.cc:121] Time spent starting KDC: real 2.573s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:20 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:20.968683 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:35171
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:35171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:21.306304 2826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:21.306617 2826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:21.306756 2826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:21.314191 2826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:21.314314 2826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:21.314391 2826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:21.314468 2826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:21.314527 2826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:21.324684 2826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:35171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:35171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:21.326188 2826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:21.327580 2826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:21.339457 2831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:21.339332 2834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:21.339380 2832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:21.340299 2826 server_base.cc:1034] running on GCE node
I20260808 13:10:21.341040 2826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:21.342439 2826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:21.343648 2826 hybrid_clock.cc:648] HybridClock initialized: now 1786194621343590 us; error 42 us; skew 500 ppm
I20260808 13:10:21.343981 2826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:21 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194621, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:21.347128 2826 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:21.348574 2826 webserver.cc:460] Webserver started at http://127.31.59.190:33731/ using document root <none> and password file <none>
I20260808 13:10:21.349215 2826 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:21.349335 2826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:21.349668 2826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:21.352591 2826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "8f74e0e7a4c04e08be31844fc9d1143c"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "9deba2663a9c4b81e00cab0c77fe59f1"
server_key_iv: "20d9fe76b3f52c2622c9f07a0e0558a3"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.353334 2826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "8f74e0e7a4c04e08be31844fc9d1143c"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "9deba2663a9c4b81e00cab0c77fe59f1"
server_key_iv: "20d9fe76b3f52c2622c9f07a0e0558a3"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.358415 2826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:10:21.361814 2841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:21.363081 2826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:21.363247 2826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "8f74e0e7a4c04e08be31844fc9d1143c"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "9deba2663a9c4b81e00cab0c77fe59f1"
server_key_iv: "20d9fe76b3f52c2622c9f07a0e0558a3"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.363427 2826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:21.381064 2826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:21.384894 2826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:21.385177 2826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:21.402979 2826 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:35171
I20260808 13:10:21.403012 2894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:35171 every 8 connection(s)
I20260808 13:10:21.404374 2826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:21.408248 2895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:21.409611 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2826
I20260808 13:10:21.409808 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:21.410446 31982 external_mini_cluster.cc:1428] Setting key b7c1884c10b661abca2681265dd473db
I20260808 13:10:21.419883 2895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c: Bootstrap starting.
I20260808 13:10:21.423596 2895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:21.424821 2895 log.cc:824] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:10:21 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:21.427804 2895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c: No bootstrap required, opened a new log
I20260808 13:10:21.432973 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:21.412414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.31.59.190:35171)
0808 13:10:21.413290 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:21.413305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:21.413348 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:10:21.415106 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:21.415128 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:21.415481 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:21.415773 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:21.417504 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:21.418845 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:21.420505 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:21.421189 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:21.421322 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:21.425965 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:21.425973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:21.425987 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:21.426029 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:21.429037 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:21.429739 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:21.429745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:21.429750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:21.429913 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:21.430298 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:21.430301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:21.430303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:21.430815 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:21.431132 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:21.431520 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 13:10:21.431931 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":380,"threads_started":1}
I20260808 13:10:21.435571 2895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } }
I20260808 13:10:21.435998 2895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:21.436096 2895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f74e0e7a4c04e08be31844fc9d1143c, State: Initialized, Role: FOLLOWER
I20260808 13:10:21.436612 2895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } }
I20260808 13:10:21.436825 2895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:21.436959 2895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:21.437107 2895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:21.439476 2895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } }
I20260808 13:10:21.439971 2895 leader_election.cc:302] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f74e0e7a4c04e08be31844fc9d1143c; no voters:
I20260808 13:10:21.440529 2895 leader_election.cc:288] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:21.440639 2902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:21.442072 2902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [term 1 LEADER]: Becoming Leader. State: Replica: 8f74e0e7a4c04e08be31844fc9d1143c, State: Running, Role: LEADER
I20260808 13:10:21.442694 2902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } }
I20260808 13:10:21.443153 2895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:21.445760 2903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } } }
I20260808 13:10:21.446187 2903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:21.446318 2904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f74e0e7a4c04e08be31844fc9d1143c. Latest consensus state: current_term: 1 leader_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f74e0e7a4c04e08be31844fc9d1143c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35171 } } }
I20260808 13:10:21.451108 2904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:21.452663 2911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:21.457763 2911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:21.466354 2911 catalog_manager.cc:1347] Generated new cluster ID: ca6475dffa0b4da38b21f35260efee78
I20260808 13:10:21.466467 2911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:21.487600 2911 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:21.489477 2911 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:21.502892 2911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f74e0e7a4c04e08be31844fc9d1143c: Generated new TSK 0
I20260808 13:10:21.503720 2911 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:21.566569 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:21.909260 2925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:21.909567 2925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:21.909685 2925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:21.916922 2925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:21.917037 2925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:21.917161 2925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:21.927420 2925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:21.928741 2925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:21.930095 2925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:21.946411 2934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:21.946995 2932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:21.947183 2930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:21.947790 2925 server_base.cc:1034] running on GCE node
I20260808 13:10:21.948344 2925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:21.949332 2925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:21.950526 2925 hybrid_clock.cc:648] HybridClock initialized: now 1786194621950476 us; error 44 us; skew 500 ppm
I20260808 13:10:21.950834 2925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:21 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194621, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:21.953949 2925 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:21.955312 2925 webserver.cc:460] Webserver started at http://127.31.59.129:44841/ using document root <none> and password file <none>
I20260808 13:10:21.955979 2925 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:21.956080 2925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:21.956331 2925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:21.959223 2925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "792cc7e4fe3142e9b0352a3bc10c51de"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "476f5a2028e51fcdc6e4acdd29c5095d"
server_key_iv: "a025839e8249194da378a5b686315bbb"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.960011 2925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "792cc7e4fe3142e9b0352a3bc10c51de"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "476f5a2028e51fcdc6e4acdd29c5095d"
server_key_iv: "a025839e8249194da378a5b686315bbb"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.965009 2925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:10:21.968618 2941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:21.970034 2925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:21.970191 2925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "792cc7e4fe3142e9b0352a3bc10c51de"
format_stamp: "Formatted at 2026-08-08 13:10:21 on dist-test-slave-49h3"
server_key: "476f5a2028e51fcdc6e4acdd29c5095d"
server_key_iv: "a025839e8249194da378a5b686315bbb"
server_key_version: "encryptionkey@0"
I20260808 13:10:21.970394 2925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:22.005573 2925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:22.010237 2925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:22.010552 2925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:22.011569 2925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:22.012959 2925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:22.013041 2925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.013176 2925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:22.013233 2925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.045564 2925 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:33973
I20260808 13:10:22.045607 3057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:33973 every 8 connection(s)
I20260808 13:10:22.046936 2925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:22.051491 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 2925
I20260808 13:10:22.051698 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:22.052350 31982 external_mini_cluster.cc:1428] Setting key 6d45700a02cf35e7ecce86f703ef2377
Aug 08 13:10:22 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194621, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:22.067292 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:22.049344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:34681 (local address 127.31.59.190:35171)
0808 13:10:22.049859 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:22.049865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:22.050848 (+ 983us) server_negotiation.cc:407] Connection header received
0808 13:10:22.052714 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:22.052719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:22.052807 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:22.052936 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:22.054724 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.055765 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.057258 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.057800 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.057930 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:22.061213 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:22.061218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:22.061222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:22.061259 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:22.063899 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:22.064809 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:22.064813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:22.064816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:22.064946 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:22.065394 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:22.065397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:22.065400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:22.065737 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:22.065856 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:22.066773 (+ 917us) server_negotiation.cc:299] Negotiation successful
0808 13:10:22.066957 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":241,"threads_started":1}
I20260808 13:10:22.068595 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:22.049970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:35171 (local address 127.31.59.129:34681)
0808 13:10:22.050683 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:22.050721 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:22.052454 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:22.053401 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:22.053410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:22.053995 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:22.054560 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:22.054576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.055888 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.055891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:22.057108 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:22.057119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.058005 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.058010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:22.058243 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:22.058860 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:22.058887 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:22.061014 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:22.064038 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:22.064047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:22.064065 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:22.064675 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:22.065044 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:22.065049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:22.065052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:22.065283 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:22.065861 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:22.065868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:22.066355 (+ 487us) client_negotiation.cc:769] Sending connection context
0808 13:10:22.066674 (+ 319us) client_negotiation.cc:240] Negotiation successful
0808 13:10:22.067224 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":301,"threads_started":1}
I20260808 13:10:22.070238 3058 heartbeater.cc:347] Connected to a master server at 127.31.59.190:35171
I20260808 13:10:22.070525 3058 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:22.071267 3058 heartbeater.cc:511] Master 127.31.59.190:35171 requested a full tablet report, sending...
I20260808 13:10:22.073576 2859 ts_manager.cc:194] Registered new tserver with Master: 792cc7e4fe3142e9b0352a3bc10c51de (127.31.59.129:33973)
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
I20260808 13:10:22.075261 2859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:34681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:22.100781 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:22.422166 3065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:22.422430 3065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:22.422533 3065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:22.429661 3065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:22.429796 3065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:22.429950 3065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:22.439603 3065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:22.440861 3065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:22.442193 3065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:22.455215 3071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:22.455186 3072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:22.455358 3074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:22.455861 3065 server_base.cc:1034] running on GCE node
I20260808 13:10:22.456472 3065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:22.457422 3065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:22.458694 3065 hybrid_clock.cc:648] HybridClock initialized: now 1786194622458639 us; error 33 us; skew 500 ppm
I20260808 13:10:22.459071 3065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:22 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194622, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:22.462203 3065 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:22.463578 3065 webserver.cc:460] Webserver started at http://127.31.59.130:44145/ using document root <none> and password file <none>
I20260808 13:10:22.464313 3065 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:22.464439 3065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:22.464728 3065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:22.467852 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "ca7c6680300e4904b32b7c5c8641b082"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "28176b8b9938630c6191263cad88e7af"
server_key_iv: "1fadd99dc94c5debdd63e9114b2b905a"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.468727 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "ca7c6680300e4904b32b7c5c8641b082"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "28176b8b9938630c6191263cad88e7af"
server_key_iv: "1fadd99dc94c5debdd63e9114b2b905a"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.473791 3065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 13:10:22.477089 3081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.478425 3065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:22.478585 3065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "ca7c6680300e4904b32b7c5c8641b082"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "28176b8b9938630c6191263cad88e7af"
server_key_iv: "1fadd99dc94c5debdd63e9114b2b905a"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.478762 3065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:22.492364 3065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:22.496479 3065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:22.496768 3065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:22.497749 3065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:22.499094 3065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:22.499173 3065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.499290 3065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:22.499357 3065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.534377 3065 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:39055
I20260808 13:10:22.534420 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:39055 every 8 connection(s)
I20260808 13:10:22.535878 3065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:22.541868 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3065
I20260808 13:10:22.542035 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:22.542481 31982 external_mini_cluster.cc:1428] Setting key 023d41a1b31249264bbb0c1687a2cd85
Aug 08 13:10:22 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194622, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:22.556341 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:22.538404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:48125 (local address 127.31.59.190:35171)
0808 13:10:22.538588 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:22.538593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:22.539784 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 13:10:22.541631 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:22.541637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:22.541716 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:22.541809 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:22.543523 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.544305 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.546585 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.547036 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.547135 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:22.550913 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:22.550919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:22.550922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:22.550954 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:22.553405 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:22.554129 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:22.554133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:22.554136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:22.554263 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:22.554696 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:22.554699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:22.554702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:22.555000 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:22.555114 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:22.555869 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 13:10:22.556093 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:10:22.557674 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:22.538877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:35171 (local address 127.31.59.130:48125)
0808 13:10:22.539609 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:22.539669 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:22.541334 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:22.542115 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:22.542127 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:22.542666 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:22.543362 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:22.543383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.544488 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.544492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:22.546449 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:22.546465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:22.547257 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:22.547264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:22.547528 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:22.548205 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:22.548232 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:22.550694 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:22.553585 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:22.553594 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:22.553610 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:22.554006 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:22.554363 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:22.554368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:22.554372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:22.554568 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:22.555116 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:22.555124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:22.555518 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 13:10:22.555793 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 13:10:22.556310 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":277,"threads_started":1}
I20260808 13:10:22.559034 3196 heartbeater.cc:347] Connected to a master server at 127.31.59.190:35171
I20260808 13:10:22.559355 3196 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:22.560231 3196 heartbeater.cc:511] Master 127.31.59.190:35171 requested a full tablet report, sending...
I20260808 13:10:22.561694 2859 ts_manager.cc:194] Registered new tserver with Master: ca7c6680300e4904b32b7c5c8641b082 (127.31.59.130:39055)
I20260808 13:10:22.562489 2859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:48125
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:22.591729 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:22.911986 3201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:22.912262 3201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:22.912357 3201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:22.919432 3201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:22.919550 3201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:22.919740 3201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:22.929232 3201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:35171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:22.930521 3201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:22.931805 3201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:22.944204 3212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:22.944273 3210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:22.944242 3208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:22.945045 3201 server_base.cc:1034] running on GCE node
I20260808 13:10:22.945724 3201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:22.946745 3201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:22.947969 3201 hybrid_clock.cc:648] HybridClock initialized: now 1786194622947896 us; error 45 us; skew 500 ppm
I20260808 13:10:22.948336 3201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:22 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194622, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:22.954761 3201 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:22.956175 3201 webserver.cc:460] Webserver started at http://127.31.59.131:33353/ using document root <none> and password file <none>
I20260808 13:10:22.956866 3201 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:22.956986 3201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:22.957280 3201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:22.960225 3201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "d76f471d28d645738b05e57cd2294c56"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "04835ca0ccf0b328c1e48a3ceb274945"
server_key_iv: "5e3347fbe867581e66af79186627bce6"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.961001 3201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "d76f471d28d645738b05e57cd2294c56"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "04835ca0ccf0b328c1e48a3ceb274945"
server_key_iv: "5e3347fbe867581e66af79186627bce6"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.966123 3201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:22.969547 3219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.970824 3201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:22.970973 3201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "d76f471d28d645738b05e57cd2294c56"
format_stamp: "Formatted at 2026-08-08 13:10:22 on dist-test-slave-49h3"
server_key: "04835ca0ccf0b328c1e48a3ceb274945"
server_key_iv: "5e3347fbe867581e66af79186627bce6"
server_key_version: "encryptionkey@0"
I20260808 13:10:22.971153 3201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:22.989329 3201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:22.993201 3201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:22.993510 3201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:22.994480 3201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:22.995795 3201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:22.995870 3201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:22.995991 3201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:22.996044 3201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:23.025892 3201 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:34781
I20260808 13:10:23.025926 3335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:34781 every 8 connection(s)
I20260808 13:10:23.027151 3201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:23.031546 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3201
I20260808 13:10:23.031706 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:23.032145 31982 external_mini_cluster.cc:1428] Setting key 2ea9768ae6da9902ebcea016c10d636f
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194622, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:23.046780 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.029560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:53951 (local address 127.31.59.190:35171)
0808 13:10:23.029766 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:23.029772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:23.030948 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 13:10:23.032795 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:23.032801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:23.032892 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:23.033004 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:23.034839 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.035740 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.037186 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.037639 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.037730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.040853 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:23.040858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:23.040861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:23.040893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:23.043481 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.044094 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.044097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.044099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.044223 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.044623 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.044626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.044628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.044968 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:23.045119 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:23.045885 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 13:10:23.046328 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:10:23.047629 3338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.030088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:35171 (local address 127.31.59.131:53951)
0808 13:10:23.030786 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:23.030830 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:23.032509 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:23.033232 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:23.033244 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:23.033881 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:23.034658 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:23.034684 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.035872 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.035876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:23.037080 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:23.037091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.037818 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.037823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:23.038012 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.038582 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:23.038609 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:23.040662 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:23.043604 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:23.043610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:23.043621 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:23.044007 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:23.044310 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:23.044313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:23.044315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:23.044540 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:23.045110 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:23.045115 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:23.045440 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 13:10:23.045800 (+ 360us) client_negotiation.cc:240] Negotiation successful
0808 13:10:23.046326 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":273,"threads_started":1}
I20260808 13:10:23.049001 3336 heartbeater.cc:347] Connected to a master server at 127.31.59.190:35171
I20260808 13:10:23.049304 3336 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:23.050236 3336 heartbeater.cc:511] Master 127.31.59.190:35171 requested a full tablet report, sending...
I20260808 13:10:23.051654 2859 ts_manager.cc:194] Registered new tserver with Master: d76f471d28d645738b05e57cd2294c56 (127.31.59.131:34781)
I20260808 13:10:23.052429 2859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:53951
I20260808 13:10:23.059491 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:10:23.078161 3058 heartbeater.cc:503] Master 127.31.59.190:35171 was elected leader, sending a full tablet report...
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:23.110932 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.098061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.31.59.190:35171)
0808 13:10:23.098295 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:23.098301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:23.098646 (+ 345us) server_negotiation.cc:407] Connection header received
0808 13:10:23.098808 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:23.098812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:23.098879 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:23.098965 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:23.100260 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.100994 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.102195 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.102609 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.102703 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.106419 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:23.106424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:23.106427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:23.106463 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:23.108695 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.109240 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.109243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.109245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.109362 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.109693 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.109696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.109698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.110016 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:23.110197 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:23.110512 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 13:10:23.110687 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:10:23.111498 2859 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39040
I20260808 13:10:23.112815 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2925
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:23.129287 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.114628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.31.59.129:33973)
0808 13:10:23.115311 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:23.115318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:23.115343 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:10:23.115401 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:23.115407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:23.115632 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:23.115755 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:23.116857 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.118019 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.119217 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.119793 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.119896 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.123910 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:23.123924 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:23.123940 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:23.123987 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:23.126936 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.127592 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.127598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.127601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.127731 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.128011 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.128014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.128016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.128368 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:23.128617 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:23.128787 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:10:23.128989 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":353,"threads_started":1}
W20260808 13:10:23.130632 3051 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54228
W20260808 13:10:23.131829 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2925: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:23.132052 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2925
I20260808 13:10:23.150754 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3065
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:23.165742 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.151589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.31.59.130:39055)
0808 13:10:23.152112 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:23.152119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:23.152146 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:10:23.152210 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:23.152216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:23.152449 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:23.152581 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:23.153776 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.154774 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.155868 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.156393 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.156480 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.160562 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:23.160575 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:23.160589 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:23.160621 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:23.163481 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.164059 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.164066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.164068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.164180 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.164437 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.164440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.164442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.164760 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:23.164995 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:23.165207 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 13:10:23.165415 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
W20260808 13:10:23.166978 3191 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50324
W20260808 13:10:23.168031 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3065: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:23.168166 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3065
I20260808 13:10:23.185925 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3201
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[2813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:23.201195 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:23.186920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.31.59.131:34781)
0808 13:10:23.187541 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:23.187547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:23.187576 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:10:23.187637 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:23.187644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:23.187849 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:23.187972 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:23.189244 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.190328 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.191510 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:23.192041 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:23.192131 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:23.196152 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:23.196164 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:23.196177 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:23.196215 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:23.199000 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.199569 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.199575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.199577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.199695 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:23.199952 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:23.199955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:23.199957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:23.200295 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:23.200513 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:23.200711 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 13:10:23.200907 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":331,"threads_started":1}
W20260808 13:10:23.202462 3331 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35366
W20260808 13:10:23.203531 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3201: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:10:23.203692 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3201
I20260808 13:10:23.219934 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 2826
I20260808 13:10:23.221086 2890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:23.327996 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 2826
2026-08-08T13:10:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5190 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[3357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[3357](info): set up 2 sockets
Aug 08 13:10:23 dist-test-slave-49h3 krb5kdc[3357](info): commencing operation
krb5kdc: starting...
W20260808 13:10:24.377104 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:24 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:24.584089 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40211
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:24.907541 3370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:24.907845 3370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:24.907938 3370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:24.915292 3370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:24.915413 3370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:24.915459 3370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:24.915517 3370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:24.915562 3370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:24.925561 3370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:24.926913 3370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:24.928241 3370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:24.939801 3375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:24.939867 3378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:24.939879 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:24.940534 3370 server_base.cc:1034] running on GCE node
I20260808 13:10:24.941404 3370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:24.942873 3370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:24.944094 3370 hybrid_clock.cc:648] HybridClock initialized: now 1786194624944037 us; error 49 us; skew 500 ppm
I20260808 13:10:24.944430 3370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:24 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194624, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:24.947477 3370 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:24.948896 3370 webserver.cc:460] Webserver started at http://127.31.59.190:43139/ using document root <none> and password file <none>
I20260808 13:10:24.949625 3370 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:24.949741 3370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:24.950006 3370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:24.952868 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "ba40161eb02d4a9d831f669b4bd229f1"
format_stamp: "Formatted at 2026-08-08 13:10:24 on dist-test-slave-49h3"
server_key: "8a153300a2b94135455cb34c6b2459a9"
server_key_iv: "53d33259ec35960a6eb536920284b9a1"
server_key_version: "encryptionkey@0"
I20260808 13:10:24.953775 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "ba40161eb02d4a9d831f669b4bd229f1"
format_stamp: "Formatted at 2026-08-08 13:10:24 on dist-test-slave-49h3"
server_key: "8a153300a2b94135455cb34c6b2459a9"
server_key_iv: "53d33259ec35960a6eb536920284b9a1"
server_key_version: "encryptionkey@0"
I20260808 13:10:24.959564 3370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:24.962881 3385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:24.964119 3370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:10:24.964277 3370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "ba40161eb02d4a9d831f669b4bd229f1"
format_stamp: "Formatted at 2026-08-08 13:10:24 on dist-test-slave-49h3"
server_key: "8a153300a2b94135455cb34c6b2459a9"
server_key_iv: "53d33259ec35960a6eb536920284b9a1"
server_key_version: "encryptionkey@0"
I20260808 13:10:24.964463 3370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:24.981689 3370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:24.986209 3370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:24.986510 3370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:25.003104 3370 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40211
I20260808 13:10:25.003125 3437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40211 every 8 connection(s)
I20260808 13:10:25.004467 3370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:25.008291 3438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:25.014626 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3370
I20260808 13:10:25.014873 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:25.015770 31982 external_mini_cluster.cc:1428] Setting key a03f192a88936b1f6f769966410e7383
I20260808 13:10:25.019579 3438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1: Bootstrap starting.
I20260808 13:10:25.023118 3438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:25.024199 3438 log.cc:824] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:25.027208 3438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1: No bootstrap required, opened a new log
Aug 08 13:10:25 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:25.037274 3438 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } }
I20260808 13:10:25.037824 3438 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:25.037950 3438 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba40161eb02d4a9d831f669b4bd229f1, State: Initialized, Role: FOLLOWER
I20260808 13:10:25.038637 3438 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } }
I20260808 13:10:25.038937 3438 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:25.039089 3438 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:25.039268 3438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:25.040552 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:25.017956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.31.59.190:40211)
0808 13:10:25.018757 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:25.018770 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:25.018813 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:10:25.020602 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:25.020626 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:25.021015 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:25.021325 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:25.023200 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.024557 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.026148 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.026808 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.026931 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:25.031447 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:25.031458 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:25.031478 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:25.031523 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:25.035775 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:25.036493 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:25.036502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:25.036512 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:25.036723 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:25.037057 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:25.037061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:25.037064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:25.037842 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:25.038142 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:25.038666 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 13:10:25.039119 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":369,"threads_started":1}
I20260808 13:10:25.042475 3438 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } }
I20260808 13:10:25.042961 3438 leader_election.cc:302] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba40161eb02d4a9d831f669b4bd229f1; no voters:
I20260808 13:10:25.043603 3438 leader_election.cc:288] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:25.043733 3444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:25.044441 3444 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [term 1 LEADER]: Becoming Leader. State: Replica: ba40161eb02d4a9d831f669b4bd229f1, State: Running, Role: LEADER
I20260808 13:10:25.044931 3444 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } }
I20260808 13:10:25.045620 3438 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:25.047734 3445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba40161eb02d4a9d831f669b4bd229f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } } }
I20260808 13:10:25.048111 3445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:25.048725 3446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba40161eb02d4a9d831f669b4bd229f1. Latest consensus state: current_term: 1 leader_uuid: "ba40161eb02d4a9d831f669b4bd229f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40161eb02d4a9d831f669b4bd229f1" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40211 } } }
I20260808 13:10:25.049019 3446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:25.050513 3450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:25.055851 3450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:25.065112 3450 catalog_manager.cc:1347] Generated new cluster ID: 251bcb6c57cf46efaa23b0043485846d
I20260808 13:10:25.065255 3450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:25.085796 3450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:25.087167 3450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:25.102881 3450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba40161eb02d4a9d831f669b4bd229f1: Generated new TSK 0
I20260808 13:10:25.103767 3450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:25.164410 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:25.517923 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:25.518218 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:25.518365 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:25.525736 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:25.525861 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:25.526027 3466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:25.535935 3466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:25.537254 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:25.538626 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:25.551749 3473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:25.551791 3475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:25.551837 3472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:25.552819 3466 server_base.cc:1034] running on GCE node
I20260808 13:10:25.553388 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:25.554286 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:25.555475 3466 hybrid_clock.cc:648] HybridClock initialized: now 1786194625555419 us; error 42 us; skew 500 ppm
I20260808 13:10:25.555773 3466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:25 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194625, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:25.558892 3466 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:25.560272 3466 webserver.cc:460] Webserver started at http://127.31.59.129:40407/ using document root <none> and password file <none>
I20260808 13:10:25.560963 3466 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:25.561092 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:25.561386 3466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:25.564198 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "4adf9cb2ee534359a7b05c4ee0f4d8ba"
format_stamp: "Formatted at 2026-08-08 13:10:25 on dist-test-slave-49h3"
server_key: "da8b3459f245cf8d3cf52389721ba48f"
server_key_iv: "3ac712670e804f461dc9cff4a25b8b15"
server_key_version: "encryptionkey@0"
I20260808 13:10:25.564952 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "4adf9cb2ee534359a7b05c4ee0f4d8ba"
format_stamp: "Formatted at 2026-08-08 13:10:25 on dist-test-slave-49h3"
server_key: "da8b3459f245cf8d3cf52389721ba48f"
server_key_iv: "3ac712670e804f461dc9cff4a25b8b15"
server_key_version: "encryptionkey@0"
I20260808 13:10:25.570295 3466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:10:25.573560 3483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:25.574802 3466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:10:25.574954 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "4adf9cb2ee534359a7b05c4ee0f4d8ba"
format_stamp: "Formatted at 2026-08-08 13:10:25 on dist-test-slave-49h3"
server_key: "da8b3459f245cf8d3cf52389721ba48f"
server_key_iv: "3ac712670e804f461dc9cff4a25b8b15"
server_key_version: "encryptionkey@0"
I20260808 13:10:25.575137 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:25.615834 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:25.619861 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:25.620146 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:25.621126 3466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:25.622550 3466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:25.622630 3466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:25.622751 3466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:25.622805 3466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:25.654156 3466 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:46059
I20260808 13:10:25.654212 3601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:46059 every 8 connection(s)
I20260808 13:10:25.655448 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:25.659447 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3466
I20260808 13:10:25.659680 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:25.660475 31982 external_mini_cluster.cc:1428] Setting key f0a11e73d86fe5a716df09a358318ea5
Aug 08 13:10:25 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194625, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:25.676810 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:25.657923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:42991 (local address 127.31.59.190:40211)
0808 13:10:25.658372 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:25.658377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:25.659465 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 13:10:25.661871 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:25.661877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:25.661955 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:25.662052 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:25.663984 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.664727 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.666838 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.667256 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.667400 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:25.671337 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:25.671342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:25.671345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:25.671379 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:25.673830 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:25.674546 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:25.674551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:25.674553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:25.674683 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:25.675096 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:25.675099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:25.675101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:25.675363 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:25.675495 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:25.676199 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 13:10:25.676440 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":226,"threads_started":1}
I20260808 13:10:25.677819 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:25.658498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40211 (local address 127.31.59.129:42991)
0808 13:10:25.659267 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:25.659336 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:25.661613 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:25.662283 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:25.662292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:25.663009 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:25.663788 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:25.663812 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.664886 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.664892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:25.666674 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:25.666697 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:25.667512 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:25.667518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:25.667795 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:25.668471 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:25.668498 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:25.671126 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:25.673961 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:25.673969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:25.673986 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:25.674424 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:25.674786 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:25.674791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:25.674795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:25.674978 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:25.675496 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:25.675503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:25.675828 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 13:10:25.676060 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 13:10:25.676599 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":277,"threads_started":1}
I20260808 13:10:25.679595 3602 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40211
I20260808 13:10:25.679896 3602 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:25.680617 3602 heartbeater.cc:511] Master 127.31.59.190:40211 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
I20260808 13:10:25.682657 3403 ts_manager.cc:194] Registered new tserver with Master: 4adf9cb2ee534359a7b05c4ee0f4d8ba (127.31.59.129:46059)
I20260808 13:10:25.684135 3403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:42991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:25.710096 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:26.043581 3608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:26.043854 3608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:26.043985 3608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:26.051576 3608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:26.051690 3608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:26.051859 3608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:26.061564 3608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:26.062783 3608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:26.064085 3608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:26.077026 3613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:26.076990 3616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:26.076918 3614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:26.077850 3608 server_base.cc:1034] running on GCE node
I20260808 13:10:26.078460 3608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:26.079413 3608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:26.080628 3608 hybrid_clock.cc:648] HybridClock initialized: now 1786194626080586 us; error 34 us; skew 500 ppm
I20260808 13:10:26.080904 3608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:26.083931 3608 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:26.085209 3608 webserver.cc:460] Webserver started at http://127.31.59.130:38877/ using document root <none> and password file <none>
I20260808 13:10:26.085867 3608 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:26.085969 3608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:26.086241 3608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:26.088990 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "86e1372754094b1795f75db0442518e2"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "c21df3079bed8d66ce379326ca4d0667"
server_key_iv: "d6ac5246756ddd76e47834a34136a251"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.089789 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "86e1372754094b1795f75db0442518e2"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "c21df3079bed8d66ce379326ca4d0667"
server_key_iv: "d6ac5246756ddd76e47834a34136a251"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.094722 3608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:10:26.098026 3624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.099310 3608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:26.099435 3608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "86e1372754094b1795f75db0442518e2"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "c21df3079bed8d66ce379326ca4d0667"
server_key_iv: "d6ac5246756ddd76e47834a34136a251"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.099608 3608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:26.127713 3608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:26.132084 3608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:26.132340 3608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:26.133268 3608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:26.134671 3608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:26.134747 3608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.134840 3608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:26.134883 3608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.164593 3608 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:38095
I20260808 13:10:26.164662 3744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:38095 every 8 connection(s)
I20260808 13:10:26.165979 3608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:26.172196 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3608
I20260808 13:10:26.172441 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:26.172984 31982 external_mini_cluster.cc:1428] Setting key e837d92db1c7a74ce41db90ce0672c4d
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:26.187068 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.168621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:34361 (local address 127.31.59.190:40211)
0808 13:10:26.168840 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.168845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.170043 (+ 1198us) server_negotiation.cc:407] Connection header received
0808 13:10:26.171831 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.171836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.171909 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.171994 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.174005 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.174989 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.177356 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.177849 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.177943 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.181593 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.181598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.181602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.181636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.184030 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.184738 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.184743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.184746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.184885 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.185303 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.185307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.185309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.185623 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.185777 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.186484 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.186723 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:10:26.188067 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.169132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40211 (local address 127.31.59.130:34361)
0808 13:10:26.169892 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:26.169938 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:26.171565 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:26.172267 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:26.172278 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:26.173053 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:26.173830 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:26.173858 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.175195 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.175201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:26.177184 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:26.177201 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.178075 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.178081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:26.178425 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.179145 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:26.179168 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:26.181354 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:26.184169 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:26.184179 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:26.184196 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:26.184614 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:26.184991 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:26.184996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:26.184999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:26.185185 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:26.185743 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:26.185758 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:26.186112 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 13:10:26.186345 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 13:10:26.186898 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":300,"threads_started":1}
I20260808 13:10:26.189587 3745 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40211
I20260808 13:10:26.189881 3745 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:26.190585 3745 heartbeater.cc:511] Master 127.31.59.190:40211 requested a full tablet report, sending...
I20260808 13:10:26.192072 3403 ts_manager.cc:194] Registered new tserver with Master: 86e1372754094b1795f75db0442518e2 (127.31.59.130:38095)
I20260808 13:10:26.192865 3403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:34361
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:26.222712 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:26.544302 3750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:26.544574 3750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:26.544678 3750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:26.552073 3750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:26.552179 3750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:26.552297 3750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:26.561955 3750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:26.563285 3750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:26.564586 3750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:26.577337 3756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:26.577369 3755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:26.577335 3758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:26.578388 3750 server_base.cc:1034] running on GCE node
I20260808 13:10:26.578986 3750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:26.580046 3750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:26.581256 3750 hybrid_clock.cc:648] HybridClock initialized: now 1786194626581205 us; error 33 us; skew 500 ppm
I20260808 13:10:26.581616 3750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:26.584772 3750 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:26.586247 3750 webserver.cc:460] Webserver started at http://127.31.59.131:34609/ using document root <none> and password file <none>
I20260808 13:10:26.586920 3750 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:26.587035 3750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:26.587324 3750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:26.590260 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "0684b5dfb02141e8a428b1c7f3581b04"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "2fc5aa7dd094e6f4e9090e5267c37ce0"
server_key_iv: "c6ea44d18ccf85b7b362fc914bb20219"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.591009 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "0684b5dfb02141e8a428b1c7f3581b04"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "2fc5aa7dd094e6f4e9090e5267c37ce0"
server_key_iv: "c6ea44d18ccf85b7b362fc914bb20219"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.596056 3750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:26.599401 3765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.600644 3750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:26.600818 3750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "0684b5dfb02141e8a428b1c7f3581b04"
format_stamp: "Formatted at 2026-08-08 13:10:26 on dist-test-slave-49h3"
server_key: "2fc5aa7dd094e6f4e9090e5267c37ce0"
server_key_iv: "c6ea44d18ccf85b7b362fc914bb20219"
server_key_version: "encryptionkey@0"
I20260808 13:10:26.601017 3750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:26.626528 3750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:26.630774 3750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:26.631058 3750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:26.632011 3750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:26.633327 3750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:26.633406 3750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.633548 3750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:26.633601 3750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:26.664741 3750 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:37617
I20260808 13:10:26.664777 3882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:37617 every 8 connection(s)
I20260808 13:10:26.666064 3750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:26.675117 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3750
I20260808 13:10:26.675310 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:26.675727 31982 external_mini_cluster.cc:1428] Setting key 05ef8057fabeccdec32324784de956ca
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:26.685240 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.668617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:45831 (local address 127.31.59.190:40211)
0808 13:10:26.668844 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.668850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.670151 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 13:10:26.671973 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.671979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.672083 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.672172 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.673684 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.674446 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.676037 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.676486 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.676598 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.680372 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.680376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.680380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.680412 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.682858 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.683453 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.683456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.683458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.683577 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.683893 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.683896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.683898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.684141 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.684254 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.684819 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.684980 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:10:26.686265 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.669185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40211 (local address 127.31.59.131:45831)
0808 13:10:26.670001 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:26.670039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:26.671680 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:26.672411 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:26.672419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:26.672941 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:26.673526 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:26.673544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.674586 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.674590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:26.675893 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:26.675906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.676737 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.676746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:26.677043 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.677727 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:26.677757 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:26.680174 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:26.682954 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:26.682960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:26.682970 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:26.683366 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:26.683648 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:26.683651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:26.683653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:26.683818 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:26.684238 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:26.684244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:26.684527 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 13:10:26.684737 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 13:10:26.685096 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":341,"threads_started":1}
I20260808 13:10:26.687145 3602 heartbeater.cc:503] Master 127.31.59.190:40211 was elected leader, sending a full tablet report...
I20260808 13:10:26.687696 3883 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40211
I20260808 13:10:26.687991 3883 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:26.688725 3883 heartbeater.cc:511] Master 127.31.59.190:40211 requested a full tablet report, sending...
I20260808 13:10:26.690215 3403 ts_manager.cc:194] Registered new tserver with Master: 0684b5dfb02141e8a428b1c7f3581b04 (127.31.59.131:37617)
I20260808 13:10:26.691180 3403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:45831
I20260808 13:10:26.691223 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:26.744154 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.729648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.31.59.129:46059)
0808 13:10:26.730316 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.730324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.730352 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:10:26.730430 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.730436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.730644 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.730781 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.732143 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.733167 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.734375 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.734953 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.735052 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.738909 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.738925 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.738941 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.738980 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.741732 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.742374 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.742381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.742383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.742510 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.742794 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.742798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.742799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.743145 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.743402 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.743587 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.743802 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":296,"threads_started":1}
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:26.761714 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.748799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.31.59.190:40211)
0808 13:10:26.749001 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.749006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.749078 (+ 72us) server_negotiation.cc:407] Connection header received
0808 13:10:26.749214 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.749218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.749293 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.749372 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.750705 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.751575 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.752861 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.753292 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.753385 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.757077 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.757082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.757085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.757121 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.759484 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.760052 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.760055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.760057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.760177 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.760461 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.760463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.760465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.760790 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.760975 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.761193 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.761367 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 13:10:26.762336 3403 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36878
I20260808 13:10:26.763286 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3466
I20260808 13:10:26.776117 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.764253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.31.59.129:46059)
0808 13:10:26.764468 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.764474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.764582 (+ 108us) server_negotiation.cc:407] Connection header received
0808 13:10:26.764743 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.764756 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.764836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.764920 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.766164 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.766868 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.768110 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.768534 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.768643 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.770947 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.770952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.770955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.770992 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.773622 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.774261 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.774267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.774271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.774405 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.774743 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.774756 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.774760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.775026 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.775253 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.775580 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.775817 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:10:26.776726 3596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:26.860559 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3466
I20260808 13:10:26.879335 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3608
Aug 08 13:10:26 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:26.894798 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.880316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.31.59.130:38095)
0808 13:10:26.880844 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.880851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.880877 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:10:26.880940 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.880947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.881174 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.881297 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:26.882533 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.883606 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.884787 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:26.885363 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:26.885498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:26.889478 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:26.889491 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:26.889507 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:26.889544 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:26.892118 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.892751 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.892757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.892760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.892874 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:26.893158 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:26.893161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:26.893163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:26.893744 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:26.893989 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:26.894211 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 13:10:26.894436 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":265,"threads_started":1}
I20260808 13:10:26.896095 3740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:26.979910 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3608
I20260808 13:10:26.997177 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3750
Aug 08 13:10:27 dist-test-slave-49h3 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:27.014210 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:26.998271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.31.59.131:37617)
0808 13:10:26.998896 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:26.998903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:26.998932 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:10:26.998991 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:26.998998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:26.999202 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:26.999328 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:27.000611 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:27.001804 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:27.003242 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:27.003834 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:27.003962 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:27.008314 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:27.008325 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:27.008340 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:27.008376 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:27.011335 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:27.012032 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:27.012041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:27.012043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:27.012179 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:27.012577 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:27.012580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:27.012582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:27.012936 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:27.013261 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:27.013597 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 13:10:27.013871 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":297,"threads_started":1}
I20260808 13:10:27.015487 3878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:27.097723 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3750
I20260808 13:10:27.114252 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3370
I20260808 13:10:27.115468 3433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:27.222680 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3370
2026-08-08T13:10:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3895 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:27 dist-test-slave-49h3 krb5kdc[3902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:27 dist-test-slave-49h3 krb5kdc[3902](info): set up 2 sockets
Aug 08 13:10:27 dist-test-slave-49h3 krb5kdc[3902](info): commencing operation
krb5kdc: starting...
W20260808 13:10:28.269212 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:28 dist-test-slave-49h3 krb5kdc[3902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:28.483218 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37197
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:28.809092 3916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:28.809392 3916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:28.809548 3916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:28.816896 3916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:28.817009 3916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:28.817085 3916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:28.817166 3916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:28.817233 3916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:28.827495 3916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:37197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:28.828902 3916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:28.830279 3916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:28.841308 3924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:28.841362 3921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:28.841391 3922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:28.842401 3916 server_base.cc:1034] running on GCE node
I20260808 13:10:28.843183 3916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:28.844581 3916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:28.845798 3916 hybrid_clock.cc:648] HybridClock initialized: now 1786194628845746 us; error 37 us; skew 500 ppm
I20260808 13:10:28.846148 3916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:28 dist-test-slave-49h3 krb5kdc[3902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:28.852442 3916 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:28.853735 3916 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:28.853857 3916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:28.854182 3916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:28.857323 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "efb71307d3de41eb88052b55af936583"
format_stamp: "Formatted at 2026-08-08 13:10:28 on dist-test-slave-49h3"
server_key: "407a1cb505f1aaec4d14b08bd026409a"
server_key_iv: "4fd2b06e2ff8acdf20f5d37530d8d294"
server_key_version: "encryptionkey@0"
I20260808 13:10:28.858186 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "efb71307d3de41eb88052b55af936583"
format_stamp: "Formatted at 2026-08-08 13:10:28 on dist-test-slave-49h3"
server_key: "407a1cb505f1aaec4d14b08bd026409a"
server_key_iv: "4fd2b06e2ff8acdf20f5d37530d8d294"
server_key_version: "encryptionkey@0"
I20260808 13:10:28.863567 3916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:10:28.867164 3930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:28.868499 3916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:10:28.868672 3916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "efb71307d3de41eb88052b55af936583"
format_stamp: "Formatted at 2026-08-08 13:10:28 on dist-test-slave-49h3"
server_key: "407a1cb505f1aaec4d14b08bd026409a"
server_key_iv: "4fd2b06e2ff8acdf20f5d37530d8d294"
server_key_version: "encryptionkey@0"
I20260808 13:10:28.868896 3916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:28.898828 3916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:28.902724 3916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:28.902989 3916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:28.921237 3916 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:37197
I20260808 13:10:28.921255 3983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:37197 every 8 connection(s)
I20260808 13:10:28.922242 3916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:28.923375 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 3916
I20260808 13:10:28.923600 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:28.924216 31982 external_mini_cluster.cc:1428] Setting key 6a50369f2fdb80c6673e9aa1fa0c6ab0
I20260808 13:10:28.926505 3984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:10:28 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:28.939664 3984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: Bootstrap starting.
I20260808 13:10:28.943974 3984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:28.945246 3984 log.cc:824] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:28.948206 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:28.926202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.31.59.190:37197)
0808 13:10:28.926944 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:28.926965 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:28.927019 (+ 54us) server_negotiation.cc:407] Connection header received
0808 13:10:28.928674 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:28.928695 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:28.929087 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:28.929404 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:28.931404 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:28.932766 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:28.934512 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:28.935241 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:28.935399 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:28.940160 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:28.940172 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:28.940193 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:28.940246 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:28.944140 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:28.944832 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:28.944841 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:28.944848 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:28.945104 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:28.945467 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:28.945473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:28.945476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:28.946069 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:28.946391 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:28.946837 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 13:10:28.947247 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":343,"threads_started":1}
I20260808 13:10:28.948277 3984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: No bootstrap required, opened a new log
I20260808 13:10:28.956620 3984 raft_consensus.cc:348] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } }
I20260808 13:10:28.957110 3984 raft_consensus.cc:374] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:28.957226 3984 raft_consensus.cc:729] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb71307d3de41eb88052b55af936583, State: Initialized, Role: FOLLOWER
I20260808 13:10:28.957863 3984 consensus_queue.cc:260] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } }
I20260808 13:10:28.958118 3984 raft_consensus.cc:388] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:28.958261 3984 raft_consensus.cc:482] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:28.958424 3984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:28.960896 3984 raft_consensus.cc:504] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } }
I20260808 13:10:28.961432 3984 leader_election.cc:302] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efb71307d3de41eb88052b55af936583; no voters:
I20260808 13:10:28.962071 3984 leader_election.cc:288] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:28.962175 3990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:28.962831 3990 raft_consensus.cc:686] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [term 1 LEADER]: Becoming Leader. State: Replica: efb71307d3de41eb88052b55af936583, State: Running, Role: LEADER
I20260808 13:10:28.963342 3990 consensus_queue.cc:237] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } }
I20260808 13:10:28.964089 3984 sys_catalog.cc:564] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:28.966086 3991 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efb71307d3de41eb88052b55af936583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } } }
I20260808 13:10:28.966177 3992 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb71307d3de41eb88052b55af936583. Latest consensus state: current_term: 1 leader_uuid: "efb71307d3de41eb88052b55af936583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb71307d3de41eb88052b55af936583" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37197 } } }
I20260808 13:10:28.968528 3992 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:28.968534 3991 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:28.969547 3996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:28.975184 3996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:28.986446 3996 catalog_manager.cc:1347] Generated new cluster ID: 41e58a00c994469a954afee6d1b7d8c0
I20260808 13:10:28.986591 3996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:28.994688 3996 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:28.995995 3996 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:29.005131 3996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: Generated new TSK 0
I20260808 13:10:29.006148 3996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:29.078316 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:29.411121 4013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:29.411430 4013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:29.411597 4013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:29.419018 4013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:29.419158 4013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:29.419332 4013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:29.429360 4013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:29.430750 4013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:29.432057 4013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:29.444561 4018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:29.444588 4019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:29.444978 4021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:29.445354 4013 server_base.cc:1034] running on GCE node
I20260808 13:10:29.446015 4013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:29.446939 4013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:29.448149 4013 hybrid_clock.cc:648] HybridClock initialized: now 1786194629448097 us; error 44 us; skew 500 ppm
I20260808 13:10:29.448462 4013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:29 dist-test-slave-49h3 krb5kdc[3902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194629, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:29.451603 4013 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:29.452692 4013 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:29.452788 4013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:29.453086 4013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:29.456128 4013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "37c56390ba3b4fb785df0d3e0447e43e"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "6e7d2a6c527adc0fefd78093a07cacd4"
server_key_iv: "610e1dcab218498cae7d5d98cb22422f"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.456959 4013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "37c56390ba3b4fb785df0d3e0447e43e"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "6e7d2a6c527adc0fefd78093a07cacd4"
server_key_iv: "610e1dcab218498cae7d5d98cb22422f"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.461972 4013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:29.465579 4027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:29.466890 4013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:29.467052 4013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "37c56390ba3b4fb785df0d3e0447e43e"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "6e7d2a6c527adc0fefd78093a07cacd4"
server_key_iv: "610e1dcab218498cae7d5d98cb22422f"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.467262 4013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:29.482764 4013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:29.487001 4013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:29.487265 4013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:29.488269 4013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:29.489744 4013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:29.489840 4013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:29.489946 4013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:29.490020 4013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:29.521495 4013 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44655
I20260808 13:10:29.521548 4142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44655 every 8 connection(s)
I20260808 13:10:29.522358 4013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:29.531982 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4013
I20260808 13:10:29.532197 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:29.532823 31982 external_mini_cluster.cc:1428] Setting key 445700467850f625c5fdaab98a5686fe
Aug 08 13:10:29 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194629, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:29.541838 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:29.524688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:46943 (local address 127.31.59.190:37197)
0808 13:10:29.525149 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:29.525153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:29.526056 (+ 903us) server_negotiation.cc:407] Connection header received
0808 13:10:29.527939 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:29.527945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:29.528029 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:29.528127 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:29.529718 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:29.530456 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:29.532016 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:29.532481 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:29.532594 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:29.536371 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:29.536376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:29.536379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:29.536410 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:29.538840 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:29.539561 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:29.539565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:29.539567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:29.539686 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:29.540093 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:29.540097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:29.540099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:29.540360 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:29.540497 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:29.541198 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 13:10:29.541428 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":233,"threads_started":1}
I20260808 13:10:29.542789 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:29.525136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37197 (local address 127.31.59.129:46943)
0808 13:10:29.525884 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:29.525924 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:29.527653 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:29.528374 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:29.528382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:29.528926 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:29.529546 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:29.529566 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:29.530604 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:29.530608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:29.531863 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:29.531876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:29.532714 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:29.532722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:29.532967 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:29.533688 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:29.533717 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:29.536159 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:29.538971 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:29.538981 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:29.538999 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:29.539437 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:29.539788 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:29.539793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:29.539798 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:29.539981 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:29.540483 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:29.540491 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:29.540828 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 13:10:29.541064 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 13:10:29.541636 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":325,"threads_started":1}
I20260808 13:10:29.544327 4144 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37197
I20260808 13:10:29.544656 4144 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:29.545542 4144 heartbeater.cc:511] Master 127.31.59.190:37197 requested a full tablet report, sending...
I20260808 13:10:29.547963 3948 ts_manager.cc:194] Registered new tserver with Master: 37c56390ba3b4fb785df0d3e0447e43e (127.31.59.129:44655)
I20260808 13:10:29.549906 3948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:46943
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:29.581952 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:29.906939 4150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:29.907236 4150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:29.907374 4150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:29.914899 4150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:29.915025 4150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:29.915181 4150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:29.925276 4150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:29.926642 4150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:29.927902 4150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:29.941941 4155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:29.943785 4156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:29.952941 4158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:29.953176 4150 server_base.cc:1034] running on GCE node
I20260808 13:10:29.954001 4150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:29.955235 4150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:29.956471 4150 hybrid_clock.cc:648] HybridClock initialized: now 1786194629956396 us; error 64 us; skew 500 ppm
I20260808 13:10:29.956887 4150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:29 dist-test-slave-49h3 krb5kdc[3902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194629, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:29.960749 4150 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:29.962077 4150 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:29.962167 4150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:29.962461 4150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:29.965624 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "31b01bbdf0b9193947c5c31eca2d1945"
server_key_iv: "f6ada2af6c495a5f4dbd07ea5070487c"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.966456 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "31b01bbdf0b9193947c5c31eca2d1945"
server_key_iv: "f6ada2af6c495a5f4dbd07ea5070487c"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.972247 4150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:29.976451 4164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:29.978022 4150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:29.978204 4150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6"
format_stamp: "Formatted at 2026-08-08 13:10:29 on dist-test-slave-49h3"
server_key: "31b01bbdf0b9193947c5c31eca2d1945"
server_key_iv: "f6ada2af6c495a5f4dbd07ea5070487c"
server_key_version: "encryptionkey@0"
I20260808 13:10:29.978451 4150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:30.031014 4150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:30.035334 4150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:30.035593 4150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:30.036553 4150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:30.037883 4150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:30.037985 4150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:30.038089 4150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:30.038159 4150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:30.068188 4150 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:36069
I20260808 13:10:30.068250 4279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:36069 every 8 connection(s)
I20260808 13:10:30.069005 4150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:30.073912 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4150
I20260808 13:10:30.074116 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:30.074553 31982 external_mini_cluster.cc:1428] Setting key 1b9a3197da9333136defe934e007336f
Aug 08 13:10:30 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194629, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:30.089394 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.071377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:48783 (local address 127.31.59.190:37197)
0808 13:10:30.071816 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.071821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.072787 (+ 966us) server_negotiation.cc:407] Connection header received
0808 13:10:30.074875 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.074881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.074969 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.075095 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:30.076987 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.078014 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.079528 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.080082 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.080203 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.083660 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:30.083666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:30.083670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:30.083707 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:30.086417 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.087238 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.087241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.087243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.087370 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.087804 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.087807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.087808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.088070 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:30.088183 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.088946 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.089125 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":205,"threads_started":1}
I20260808 13:10:30.090834 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.071929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37197 (local address 127.31.59.130:48783)
0808 13:10:30.072630 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.072666 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.074580 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.075336 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.075348 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.076048 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:30.076815 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.076834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.078134 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.078138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.079388 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.079401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.080289 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.080294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.080482 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.081121 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:30.081144 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:30.083438 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:30.086550 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:30.086559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:30.086575 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:30.087142 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:30.087464 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:30.087469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:30.087472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:30.087693 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:30.088195 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:30.088202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:30.088599 (+ 397us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.088889 (+ 290us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.089402 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":264,"threads_started":1}
I20260808 13:10:30.092314 4280 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37197
I20260808 13:10:30.092608 4280 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:30.093226 4280 heartbeater.cc:511] Master 127.31.59.190:37197 requested a full tablet report, sending...
I20260808 13:10:30.094694 3948 ts_manager.cc:194] Registered new tserver with Master: dd49322d0da24abfbbb0e04bb0cb0bf6 (127.31.59.130:36069)
I20260808 13:10:30.095484 3948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:48783
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:10:30.125697 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:30.449685 4287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:30.449992 4287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:30.450146 4287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:30.457486 4287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:30.457616 4287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:30.457779 4287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:30.467900 4287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:30.469218 4287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:30.470674 4287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:30.483219 4293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:30.483286 4296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:30.483325 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:30.484174 4287 server_base.cc:1034] running on GCE node
I20260808 13:10:30.484768 4287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:30.485699 4287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:30.486901 4287 hybrid_clock.cc:648] HybridClock initialized: now 1786194630486846 us; error 45 us; skew 500 ppm
I20260808 13:10:30.487200 4287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:30 dist-test-slave-49h3 krb5kdc[3902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194630, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:30.490248 4287 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:30.491320 4287 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:30.491423 4287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:30.491715 4287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:30.494632 4287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "966889c9f55b4587a13d6b63e6fb57cc"
format_stamp: "Formatted at 2026-08-08 13:10:30 on dist-test-slave-49h3"
server_key: "05b1fc3604c5acbc39f6bf3ef97d0e12"
server_key_iv: "e1f458748b15f61733b2fc4b700bbc1d"
server_key_version: "encryptionkey@0"
I20260808 13:10:30.495430 4287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "966889c9f55b4587a13d6b63e6fb57cc"
format_stamp: "Formatted at 2026-08-08 13:10:30 on dist-test-slave-49h3"
server_key: "05b1fc3604c5acbc39f6bf3ef97d0e12"
server_key_iv: "e1f458748b15f61733b2fc4b700bbc1d"
server_key_version: "encryptionkey@0"
I20260808 13:10:30.500368 4287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:10:30.503822 4302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:30.505079 4287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:10:30.505230 4287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "966889c9f55b4587a13d6b63e6fb57cc"
format_stamp: "Formatted at 2026-08-08 13:10:30 on dist-test-slave-49h3"
server_key: "05b1fc3604c5acbc39f6bf3ef97d0e12"
server_key_iv: "e1f458748b15f61733b2fc4b700bbc1d"
server_key_version: "encryptionkey@0"
I20260808 13:10:30.505435 4287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:30.523165 4287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:30.528036 4287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:30.528263 4287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:30.529161 4287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:30.530481 4287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:30.530568 4287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:30.530644 4287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:30.530700 4287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:30.553570 4144 heartbeater.cc:503] Master 127.31.59.190:37197 was elected leader, sending a full tablet report...
I20260808 13:10:30.559975 4287 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:43387
I20260808 13:10:30.560022 4423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:43387 every 8 connection(s)
I20260808 13:10:30.560762 4287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:30.567997 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4287
I20260808 13:10:30.568182 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:30.568585 31982 external_mini_cluster.cc:1428] Setting key 2f9bd61c2eef869613dc9514d3572438
Aug 08 13:10:30 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194630, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:30.579850 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.563124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:36123 (local address 127.31.59.190:37197)
0808 13:10:30.563301 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.563305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.564522 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 13:10:30.566440 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.566445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.566520 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.566595 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:30.568039 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.568872 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.570805 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.571195 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.571308 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.574782 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:30.574787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:30.574790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:30.574824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:30.577310 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.577989 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.577993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.577995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.578119 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.578439 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.578442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.578444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.578702 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:30.578823 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.579420 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.579589 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:10:30.580797 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.563611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37197 (local address 127.31.59.131:36123)
0808 13:10:30.564366 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.564416 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.566180 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.566773 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.566780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.567298 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:30.567868 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.567889 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.569029 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.569035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.570604 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.570618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.571409 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.571416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.571679 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.572354 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:30.572377 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:30.574575 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:30.577421 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:30.577428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:30.577439 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:30.577895 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:30.578194 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:30.578197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:30.578200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:30.578359 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:30.578804 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:30.578809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:30.579105 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.579323 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.579721 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":300,"threads_started":1}
I20260808 13:10:30.582157 4424 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37197
I20260808 13:10:30.582474 4424 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:30.583271 4424 heartbeater.cc:511] Master 127.31.59.190:37197 requested a full tablet report, sending...
I20260808 13:10:30.584649 3948 ts_manager.cc:194] Registered new tserver with Master: 966889c9f55b4587a13d6b63e6fb57cc (127.31.59.131:43387)
I20260808 13:10:30.585417 3948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:36123
I20260808 13:10:30.595690 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:10:30.611835 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.600012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.31.59.190:37197)
0808 13:10:30.600217 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.600223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.600630 (+ 407us) server_negotiation.cc:407] Connection header received
0808 13:10:30.600769 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.600772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.600837 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.600911 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:30.602261 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.602972 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.604108 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.604517 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.604607 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.606687 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:30.606692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:30.606695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:30.606725 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:30.609217 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.609909 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.609913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.609916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.610050 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.610415 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.610418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.610420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.610740 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:30.610990 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.611298 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.611512 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:10:30.615657 3948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:30.622378 3948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:30.644783 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.635753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.31.59.130:36069)
0808 13:10:30.636722 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.636731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.637482 (+ 751us) server_negotiation.cc:407] Connection header received
0808 13:10:30.637585 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.637592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.637821 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.638002 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:30.638527 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639528 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.642093 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.643319 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.643456 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.643655 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.644284 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.644460 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":674,"threads_started":1}
I20260808 13:10:30.645313 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.635158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:36069 (local address 127.0.0.1:41988)
0808 13:10:30.637159 (+ 2001us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.637176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.637304 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.638000 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.638004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.638027 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:30.638369 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.638377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639693 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.639697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.641957 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.641969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.643528 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.643534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.643722 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.643774 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.643890 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.644064 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1876,"mutex_wait_us":61,"thread_start_us":1735,"threads_started":1}
I20260808 13:10:30.645349 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.634680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.31.59.131:43387)
0808 13:10:30.635615 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.635625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.636281 (+ 656us) server_negotiation.cc:407] Connection header received
0808 13:10:30.636386 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.636393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.636632 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.636832 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:30.637873 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639477 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.642735 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.643748 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.643872 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.644079 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.644861 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.645016 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":590,"threads_started":1}
I20260808 13:10:30.645521 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.634069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.31.59.129:44655)
0808 13:10:30.636411 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.636419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.636442 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:10:30.636524 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.636531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.636713 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.637706 (+ 993us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:30.637930 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639483 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.642624 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.644004 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.644879 (+ 875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.645051 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.645113 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.645233 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2188,"thread_start_us":2026,"threads_started":1}
I20260808 13:10:30.646791 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.633910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44655 (local address 127.0.0.1:44478)
0808 13:10:30.634746 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.634803 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.634977 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.636940 (+ 1963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.636951 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.636979 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:30.637465 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.637489 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639634 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.639640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.642445 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.642464 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.644285 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.644292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.644554 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.644586 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.644665 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.644766 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":201,"thread_start_us":416,"threads_started":1}
I20260808 13:10:30.646836 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.634053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:43387 (local address 127.0.0.1:48216)
0808 13:10:30.636022 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.636039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.636158 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.637118 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.637122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.637151 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:30.637658 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.637670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.639634 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.639640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.642591 (+ 2951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.642603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.644259 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.644266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.644472 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.644503 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.644589 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.644703 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1854,"mutex_wait_us":25,"thread_start_us":1725,"threads_started":1}
I20260808 13:10:30.649523 4213 tablet_service.cc:1467] Processing CreateTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff (DEFAULT_TABLE table=test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:30.650036 4354 tablet_service.cc:1467] Processing CreateTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff (DEFAULT_TABLE table=test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:30.650400 4075 tablet_service.cc:1467] Processing CreateTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff (DEFAULT_TABLE table=test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:30.651031 4213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0198ca54d6fb4ee2bf363f4f2c698dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:30.651131 4354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0198ca54d6fb4ee2bf363f4f2c698dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:30.651628 4075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0198ca54d6fb4ee2bf363f4f2c698dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:30.662401 4442 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Bootstrap starting.
I20260808 13:10:30.665169 4443 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Bootstrap starting.
I20260808 13:10:30.666077 4444 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Bootstrap starting.
I20260808 13:10:30.666875 4442 tablet_bootstrap.cc:654] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:30.668160 4442 log.cc:824] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:30.669508 4443 tablet_bootstrap.cc:654] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:30.669656 4444 tablet_bootstrap.cc:654] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:30.670624 4444 log.cc:824] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:30.670708 4443 log.cc:824] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:30.671572 4442 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: No bootstrap required, opened a new log
I20260808 13:10:30.671821 4442 ts_tablet_manager.cc:1397] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:10:30.673377 4443 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: No bootstrap required, opened a new log
I20260808 13:10:30.673684 4443 ts_tablet_manager.cc:1397] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:10:30.673743 4444 tablet_bootstrap.cc:492] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: No bootstrap required, opened a new log
I20260808 13:10:30.673996 4444 ts_tablet_manager.cc:1397] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 13:10:30.681192 4443 raft_consensus.cc:348] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.681659 4443 raft_consensus.cc:374] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:30.681766 4443 raft_consensus.cc:729] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd49322d0da24abfbbb0e04bb0cb0bf6, State: Initialized, Role: FOLLOWER
I20260808 13:10:30.682446 4443 consensus_queue.cc:260] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.682785 4442 raft_consensus.cc:348] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.683269 4442 raft_consensus.cc:374] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:30.683648 4280 heartbeater.cc:503] Master 127.31.59.190:37197 was elected leader, sending a full tablet report...
I20260808 13:10:30.684151 4443 ts_tablet_manager.cc:1428] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 13:10:30.683406 4442 raft_consensus.cc:729] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 966889c9f55b4587a13d6b63e6fb57cc, State: Initialized, Role: FOLLOWER
I20260808 13:10:30.684168 4444 raft_consensus.cc:348] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.684558 4444 raft_consensus.cc:374] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:30.684669 4444 raft_consensus.cc:729] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c56390ba3b4fb785df0d3e0447e43e, State: Initialized, Role: FOLLOWER
I20260808 13:10:30.684983 4442 consensus_queue.cc:260] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.685226 4444 consensus_queue.cc:260] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.687546 4444 ts_tablet_manager.cc:1428] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260808 13:10:30.689141 4442 ts_tablet_manager.cc:1428] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260808 13:10:30.689224 4424 heartbeater.cc:503] Master 127.31.59.190:37197 was elected leader, sending a full tablet report...
I20260808 13:10:30.734908 4449 raft_consensus.cc:482] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:30.735280 4449 raft_consensus.cc:504] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.736976 4449 leader_election.cc:288] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37c56390ba3b4fb785df0d3e0447e43e (127.31.59.129:44655), 966889c9f55b4587a13d6b63e6fb57cc (127.31.59.131:43387)
I20260808 13:10:30.743649 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.737327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:50443 (local address 127.31.59.131:43387)
0808 13:10:30.737555 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.737562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.738397 (+ 835us) server_negotiation.cc:407] Connection header received
0808 13:10:30.738555 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.738559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.738626 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.738705 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:30.739323 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.740038 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.741940 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.742721 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.742890 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.742962 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.743264 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.743396 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:10:30.743654 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.737572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:39075 (local address 127.31.59.129:44655)
0808 13:10:30.737813 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.737820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.738266 (+ 446us) server_negotiation.cc:407] Connection header received
0808 13:10:30.738485 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.738491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.738567 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.738632 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:30.739204 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.739948 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.741930 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.742701 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.742841 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.742919 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.743283 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.743410 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:10:30.743832 4453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.737575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:43387 (local address 127.31.59.130:50443)
0808 13:10:30.738294 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.738308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.738425 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.738739 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.738756 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.738810 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:30.739170 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.739183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.740176 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.740182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.741778 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.741791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.742962 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.742967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.743101 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.743139 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.743214 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.743368 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":29,"spinlock_wait_cycles":3456,"thread_start_us":449,"threads_started":1}
I20260808 13:10:30.744081 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.737333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44655 (local address 127.31.59.130:39075)
0808 13:10:30.738152 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:30.738169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:30.738323 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:30.738671 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:30.738675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:30.738700 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:30.739066 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.739076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.740075 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.740080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.741781 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:30.741794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.742928 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.742934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:30.743088 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.743151 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 13:10:30.743241 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 13:10:30.743396 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":143,"thread_start_us":459,"threads_started":1}
I20260808 13:10:30.744814 4375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0198ca54d6fb4ee2bf363f4f2c698dff" candidate_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "966889c9f55b4587a13d6b63e6fb57cc" is_pre_election: true
I20260808 13:10:30.745097 4095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0198ca54d6fb4ee2bf363f4f2c698dff" candidate_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37c56390ba3b4fb785df0d3e0447e43e" is_pre_election: true
I20260808 13:10:30.745507 4375 raft_consensus.cc:2454] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd49322d0da24abfbbb0e04bb0cb0bf6 in term 0.
I20260808 13:10:30.745653 4095 raft_consensus.cc:2454] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd49322d0da24abfbbb0e04bb0cb0bf6 in term 0.
I20260808 13:10:30.746263 4168 leader_election.cc:302] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 37c56390ba3b4fb785df0d3e0447e43e, 966889c9f55b4587a13d6b63e6fb57cc, dd49322d0da24abfbbb0e04bb0cb0bf6; no voters:
I20260808 13:10:30.746716 4449 raft_consensus.cc:2781] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:30.746876 4449 raft_consensus.cc:482] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:30.746975 4449 raft_consensus.cc:3037] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:30.749178 4449 raft_consensus.cc:504] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.749818 4449 leader_election.cc:288] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [CANDIDATE]: Term 1 election: Requested vote from peers 37c56390ba3b4fb785df0d3e0447e43e (127.31.59.129:44655), 966889c9f55b4587a13d6b63e6fb57cc (127.31.59.131:43387)
I20260808 13:10:30.750294 4095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0198ca54d6fb4ee2bf363f4f2c698dff" candidate_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37c56390ba3b4fb785df0d3e0447e43e"
I20260808 13:10:30.750315 4375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0198ca54d6fb4ee2bf363f4f2c698dff" candidate_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "966889c9f55b4587a13d6b63e6fb57cc"
I20260808 13:10:30.750475 4095 raft_consensus.cc:3037] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:30.750499 4375 raft_consensus.cc:3037] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:30.752650 4095 raft_consensus.cc:2454] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd49322d0da24abfbbb0e04bb0cb0bf6 in term 1.
I20260808 13:10:30.752661 4375 raft_consensus.cc:2454] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd49322d0da24abfbbb0e04bb0cb0bf6 in term 1.
I20260808 13:10:30.753062 4167 leader_election.cc:302] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37c56390ba3b4fb785df0d3e0447e43e, dd49322d0da24abfbbb0e04bb0cb0bf6; no voters:
I20260808 13:10:30.753397 4449 raft_consensus.cc:2781] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:30.753917 4449 raft_consensus.cc:686] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 1 LEADER]: Becoming Leader. State: Replica: dd49322d0da24abfbbb0e04bb0cb0bf6, State: Running, Role: LEADER
I20260808 13:10:30.754405 4449 consensus_queue.cc:237] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } }
I20260808 13:10:30.759284 3948 catalog_manager.cc:5495] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd49322d0da24abfbbb0e04bb0cb0bf6 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd49322d0da24abfbbb0e04bb0cb0bf6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36069 } health_report { overall_health: HEALTHY } } }
W20260808 13:10:30.774549 4145 tablet.cc:2369] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:30.801416 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.796281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.31.59.130:36069)
0808 13:10:30.796492 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.796497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.796515 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:10:30.796701 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.796705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.796790 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.796916 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:30.797242 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.798006 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.799400 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.799804 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.799904 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.800391 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:30.800929 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:30.801048 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.801123 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.801198 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:10:30.809562 4375 raft_consensus.cc:1261] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 1 FOLLOWER]: Refusing update from remote peer dd49322d0da24abfbbb0e04bb0cb0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:30.809580 4095 raft_consensus.cc:1261] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 1 FOLLOWER]: Refusing update from remote peer dd49322d0da24abfbbb0e04bb0cb0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:30.810451 4454 consensus_queue.cc:1035] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c56390ba3b4fb785df0d3e0447e43e" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:30.810842 4449 consensus_queue.cc:1035] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "966889c9f55b4587a13d6b63e6fb57cc" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 43387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:10:30.812162 4425 tablet.cc:2369] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:30.820856 4282 tablet.cc:2369] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:30.822252 4457 mvcc.cc:205] Tried to move back new op lower bound from 7316253207786299392 to 7316253207574089728. Current Snapshot: MvccSnapshot[applied={T|T < 7316253207786299392}]
I20260808 13:10:30.836694 3948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:30.836990 3948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:30.840718 3948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efb71307d3de41eb88052b55af936583: Sending DeleteTablet for 3 replicas of tablet 0198ca54d6fb4ee2bf363f4f2c698dff
I20260808 13:10:30.842005 4075 tablet_service.cc:1514] Processing DeleteTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:30 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:44478
I20260808 13:10:30.842103 4354 tablet_service.cc:1514] Processing DeleteTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:30 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:48216
I20260808 13:10:30.843148 4213 tablet_service.cc:1514] Processing DeleteTablet for tablet 0198ca54d6fb4ee2bf363f4f2c698dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:30 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:41988
I20260808 13:10:30.843966 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4013
I20260808 13:10:30.853179 4468 tablet_replica.cc:330] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: stopping tablet replica
I20260808 13:10:30.853863 4468 raft_consensus.cc:2229] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:30.854527 4468 raft_consensus.cc:2258] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:30.856173 4470 tablet_replica.cc:330] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: stopping tablet replica
I20260808 13:10:30.857559 4469 tablet_replica.cc:330] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: stopping tablet replica
I20260808 13:10:30.858058 4469 raft_consensus.cc:2229] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:30.858117 4468 ts_tablet_manager.cc:1905] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:30.858284 4470 raft_consensus.cc:2229] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:30.858608 4469 raft_consensus.cc:2258] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:30.859588 4470 raft_consensus.cc:2258] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:30.860965 4469 ts_tablet_manager.cc:1905] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:30.861649 4470 ts_tablet_manager.cc:1905] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:30.868168 4469 ts_tablet_manager.cc:1918] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:30.868458 4469 log.cc:1196] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/0198ca54d6fb4ee2bf363f4f2c698dff
I20260808 13:10:30.869107 4469 ts_tablet_manager.cc:1939] T 0198ca54d6fb4ee2bf363f4f2c698dff P 966889c9f55b4587a13d6b63e6fb57cc: Deleting consensus metadata
I20260808 13:10:30.869369 4470 ts_tablet_manager.cc:1918] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 08 13:10:30 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:30.869717 4470 log.cc:1196] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/0198ca54d6fb4ee2bf363f4f2c698dff
I20260808 13:10:30.870398 4470 ts_tablet_manager.cc:1939] T 0198ca54d6fb4ee2bf363f4f2c698dff P dd49322d0da24abfbbb0e04bb0cb0bf6: Deleting consensus metadata
I20260808 13:10:30.871268 3935 catalog_manager.cc:4841] TS 966889c9f55b4587a13d6b63e6fb57cc (127.31.59.131:43387): tablet 0198ca54d6fb4ee2bf363f4f2c698dff (table test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]) successfully deleted
I20260808 13:10:30.872298 3934 catalog_manager.cc:4841] TS dd49322d0da24abfbbb0e04bb0cb0bf6 (127.31.59.130:36069): tablet 0198ca54d6fb4ee2bf363f4f2c698dff (table test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]) successfully deleted
I20260808 13:10:30.874197 4468 ts_tablet_manager.cc:1918] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:30.874378 4468 log.cc:1196] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/0198ca54d6fb4ee2bf363f4f2c698dff
I20260808 13:10:30.875003 4468 ts_tablet_manager.cc:1939] T 0198ca54d6fb4ee2bf363f4f2c698dff P 37c56390ba3b4fb785df0d3e0447e43e: Deleting consensus metadata
I20260808 13:10:30.876466 3934 catalog_manager.cc:4841] TS 37c56390ba3b4fb785df0d3e0447e43e (127.31.59.129:44655): tablet 0198ca54d6fb4ee2bf363f4f2c698dff (table test-table [id=f6506f69456b44f9bd7f0a4b36cfdac4]) successfully deleted
I20260808 13:10:30.879630 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:30.845495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.31.59.129:44655)
0808 13:10:30.845682 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:30.845689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:30.855492 (+ 9803us) server_negotiation.cc:407] Connection header received
0808 13:10:30.855684 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:30.855689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:30.855775 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:30.855868 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:30.858539 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.859636 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.863829 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:30.864456 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:30.864553 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:30.872154 (+ 7601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:30.872171 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:30.872195 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:30.872256 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:30.876431 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.877274 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.877285 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.877288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.877490 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:30.877864 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:30.877869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:30.877872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:30.878362 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:30.878662 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:30.878966 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 13:10:30.879198 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":62}
I20260808 13:10:30.880296 4137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:30.986225 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4013
I20260808 13:10:31.016512 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4150
Aug 08 13:10:31 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:31.033270 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:31.017277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.31.59.130:36069)
0808 13:10:31.017484 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:31.017490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:31.017516 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:10:31.017744 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:31.017748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:31.017865 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:31.017962 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:31.019569 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:31.020396 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:31.022328 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:31.022771 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:31.022867 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:31.027580 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:31.027598 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:31.027613 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:31.027650 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:31.030526 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:31.031175 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:31.031182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:31.031184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:31.031339 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:31.031712 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:31.031716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:31.031718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:31.032101 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:31.032449 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:31.032698 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 13:10:31.032924 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:10:31.033923 4274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:31.149863 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4150
I20260808 13:10:31.169648 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4287
Aug 08 13:10:31 dist-test-slave-49h3 krb5kdc[3902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:31.190930 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:31.170438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.31.59.131:43387)
0808 13:10:31.170641 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:31.170649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:31.170702 (+ 53us) server_negotiation.cc:407] Connection header received
0808 13:10:31.170906 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:31.170910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:31.170974 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:31.171062 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:31.172420 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:31.173202 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:31.174629 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:31.175023 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:31.175135 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:31.185375 (+ 10240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:31.185394 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:31.185411 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:31.185490 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:31.188111 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:31.188808 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:31.188816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:31.188818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:31.188950 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:31.189319 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:31.189323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:31.189325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:31.189781 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:31.190067 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:31.190368 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:10:31.190607 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:10:31.191594 4419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:31.280551 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4287
I20260808 13:10:31.298223 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 3916
I20260808 13:10:31.299403 3978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:31.422291 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 3916
2026-08-08T13:10:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4199 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T13:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:31Z Disabled control of system clock
I20260808 13:10:31.497073 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34719
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:43399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:10:31.830756 4480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:31.831033 4480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:31.831116 4480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:31.838435 4480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:31.838552 4480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:31.838589 4480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:31.838652 4480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:31.838696 4480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:31.848084 4480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:34719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:31.849388 4480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:31.851187 4480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:31.862828 4485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:31.862790 4488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:31.862869 4486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:31.863916 4480 server_base.cc:1034] running on GCE node
I20260808 13:10:31.864694 4480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:31.866158 4480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:31.867372 4480 hybrid_clock.cc:648] HybridClock initialized: now 1786194631867322 us; error 35 us; skew 500 ppm
I20260808 13:10:31.867712 4480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:31.870105 4480 webserver.cc:460] Webserver started at http://127.31.59.190:38287/ using document root <none> and password file <none>
I20260808 13:10:31.870784 4480 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:31.870873 4480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:31.871156 4480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:31.874117 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad"
format_stamp: "Formatted at 2026-08-08 13:10:31 on dist-test-slave-49h3"
server_key: "d1a3a8628fe12126cf5b046c1398cfe6"
server_key_iv: "7ec58bcfcc1c118885a398bbe1f02896"
server_key_version: "encryptionkey@0"
I20260808 13:10:31.874878 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad"
format_stamp: "Formatted at 2026-08-08 13:10:31 on dist-test-slave-49h3"
server_key: "d1a3a8628fe12126cf5b046c1398cfe6"
server_key_iv: "7ec58bcfcc1c118885a398bbe1f02896"
server_key_version: "encryptionkey@0"
I20260808 13:10:31.880002 4480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:31.883423 4495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:31.884687 4480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:31.884856 4480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad"
format_stamp: "Formatted at 2026-08-08 13:10:31 on dist-test-slave-49h3"
server_key: "d1a3a8628fe12126cf5b046c1398cfe6"
server_key_iv: "7ec58bcfcc1c118885a398bbe1f02896"
server_key_version: "encryptionkey@0"
I20260808 13:10:31.885061 4480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:31.890743 4480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:31.891503 4480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:31.891793 4480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:31.909289 4480 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:34719
I20260808 13:10:31.909309 4550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:34719 every 8 connection(s)
I20260808 13:10:31.910656 4480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:31.914306 4551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:31.918334 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4480
I20260808 13:10:31.918519 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:31.919196 31982 external_mini_cluster.cc:1428] Setting key fb898248a5cb0b0ce5712e4639b2e5cc
I20260808 13:10:31.926586 4551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: Bootstrap starting.
I20260808 13:10:31.929273 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:31.921291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.31.59.190:34719)
0808 13:10:31.922454 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:31.922468 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:31.922526 (+ 58us) server_negotiation.cc:407] Connection header received
0808 13:10:31.924968 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:31.925000 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:31.925576 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:31.926009 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:31.927091 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:31.927100 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:31.927115 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:31.927133 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:31.927223 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:31.927303 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:31.927549 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:31.927684 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 13:10:31.927959 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":422,"threads_started":1}
I20260808 13:10:31.930644 4551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:31.931979 4551 log.cc:824] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:31.934823 4551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: No bootstrap required, opened a new log
I20260808 13:10:31.944114 4551 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } }
I20260808 13:10:31.944489 4551 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:31.944591 4551 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a6d83b78b14bbdae7ca922dc05c2ad, State: Initialized, Role: FOLLOWER
I20260808 13:10:31.945209 4551 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } }
I20260808 13:10:31.945425 4551 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:31.945577 4551 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:31.945724 4551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:31.948110 4551 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } }
I20260808 13:10:31.948603 4551 leader_election.cc:302] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4a6d83b78b14bbdae7ca922dc05c2ad; no voters:
I20260808 13:10:31.949220 4551 leader_election.cc:288] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:31.949344 4556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:31.950107 4556 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [term 1 LEADER]: Becoming Leader. State: Replica: c4a6d83b78b14bbdae7ca922dc05c2ad, State: Running, Role: LEADER
I20260808 13:10:31.950686 4556 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } }
I20260808 13:10:31.951336 4551 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:31.953572 4557 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } } }
I20260808 13:10:31.954011 4557 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:31.953877 4558 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4a6d83b78b14bbdae7ca922dc05c2ad. Latest consensus state: current_term: 1 leader_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a6d83b78b14bbdae7ca922dc05c2ad" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34719 } } }
I20260808 13:10:31.954205 4558 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:31.955435 4563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:31.960459 4563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:31.970311 4563 catalog_manager.cc:1347] Generated new cluster ID: 6e7da496fe3446a5affb0312e6ebc27d
I20260808 13:10:31.970435 4563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:31.987300 4563 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:31.987725 4563 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:31.993979 4563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: Generated new TSK 0
I20260808 13:10:31.994786 4563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:10:32.002450 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--builtin_ntp_servers=127.31.59.148:43399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:10:32.361594 4576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:32.361866 4576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:32.361996 4576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:32.369220 4576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:32.369316 4576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:32.369473 4576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:32.378525 4576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:32.379804 4576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:32.381661 4576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:32.394654 4581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:32.394713 4582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:32.394876 4584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:32.395524 4576 server_base.cc:1034] running on GCE node
I20260808 13:10:32.396108 4576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:32.396997 4576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:32.398207 4576 hybrid_clock.cc:648] HybridClock initialized: now 1786194632398141 us; error 52 us; skew 500 ppm
I20260808 13:10:32.398561 4576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:32.400792 4576 webserver.cc:460] Webserver started at http://127.31.59.129:35827/ using document root <none> and password file <none>
I20260808 13:10:32.401528 4576 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:32.401645 4576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:32.401966 4576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:32.404798 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "1e0f357b83c5d44842e652ab3a3b631e"
server_key_iv: "b9395465fc4d31d931eea32eebb2068b"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.405634 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "1e0f357b83c5d44842e652ab3a3b631e"
server_key_iv: "b9395465fc4d31d931eea32eebb2068b"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.410779 4576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:10:32.414430 4591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.415786 4576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:32.415949 4576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "1e0f357b83c5d44842e652ab3a3b631e"
server_key_iv: "b9395465fc4d31d931eea32eebb2068b"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.416178 4576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:32.421993 4576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:32.422871 4576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:32.423159 4576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:32.424136 4576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:32.425678 4576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:32.425781 4576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.425910 4576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:32.425982 4576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.457696 4576 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:42185
I20260808 13:10:32.457762 4708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:42185 every 8 connection(s)
I20260808 13:10:32.459205 4576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:32.467293 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:32.461740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:52099 (local address 127.31.59.190:34719)
0808 13:10:32.462197 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:32.462201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:32.463369 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 13:10:32.465064 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:32.465069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:32.465142 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:32.465242 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:32.466545 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:32.466551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:32.466554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:32.466570 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:32.466613 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:32.466624 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:32.466737 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:32.466931 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 13:10:32.466997 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":244,"threads_started":1}
I20260808 13:10:32.468139 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:32.462310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34719 (local address 127.31.59.129:52099)
0808 13:10:32.463207 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:32.463250 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:32.464799 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:32.465531 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:32.465539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:32.465627 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:32.466263 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:32.466265 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:32.466301 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:32.466381 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:32.466710 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:32.466721 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:32.466729 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:10:32.466989 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 13:10:32.467227 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":261,"threads_started":1}
I20260808 13:10:32.469051 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4576
I20260808 13:10:32.469256 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:32.469617 4710 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34719
I20260808 13:10:32.469888 4710 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:32.470026 31982 external_mini_cluster.cc:1428] Setting key 34251f51a9effe6268cc788110114934
I20260808 13:10:32.470471 4710 heartbeater.cc:511] Master 127.31.59.190:34719 requested a full tablet report, sending...
I20260808 13:10:32.472350 4512 ts_manager.cc:194] Registered new tserver with Master: 1f7e419e83034ca1b76ee9d6ca21a2c6 (127.31.59.129:42185)
I20260808 13:10:32.472579 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--builtin_ntp_servers=127.31.59.148:43399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:10:32.796271 4715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:32.796558 4715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:32.796679 4715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:32.804096 4715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:32.804198 4715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:32.804339 4715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:32.813555 4715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:32.814888 4715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:32.816707 4715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:32.830055 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:32.829934 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:32.829926 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:32.830979 4715 server_base.cc:1034] running on GCE node
I20260808 13:10:32.831570 4715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:32.832522 4715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:32.833716 4715 hybrid_clock.cc:648] HybridClock initialized: now 1786194632833656 us; error 52 us; skew 500 ppm
I20260808 13:10:32.834020 4715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:32.836221 4715 webserver.cc:460] Webserver started at http://127.31.59.130:37119/ using document root <none> and password file <none>
I20260808 13:10:32.836899 4715 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:32.837003 4715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:32.837318 4715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:32.840183 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "0c9e2835272f477dac312e3a02188a39"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "9300246ae467d7fdeee1cb198f648521"
server_key_iv: "729721bffb96f258283f6fa5437ba4e3"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.840960 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "0c9e2835272f477dac312e3a02188a39"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "9300246ae467d7fdeee1cb198f648521"
server_key_iv: "729721bffb96f258283f6fa5437ba4e3"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.845963 4715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:32.849210 4730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.850495 4715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:32.850649 4715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "0c9e2835272f477dac312e3a02188a39"
format_stamp: "Formatted at 2026-08-08 13:10:32 on dist-test-slave-49h3"
server_key: "9300246ae467d7fdeee1cb198f648521"
server_key_iv: "729721bffb96f258283f6fa5437ba4e3"
server_key_version: "encryptionkey@0"
I20260808 13:10:32.850858 4715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:32.856333 4715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:32.857139 4715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:32.857386 4715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:32.858321 4715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:32.859709 4715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:32.859814 4715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.859922 4715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:32.859992 4715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:32.890223 4715 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:43197
I20260808 13:10:32.890285 4844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:43197 every 8 connection(s)
I20260808 13:10:32.891541 4715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:32.892596 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4715
I20260808 13:10:32.892776 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:32.893347 31982 external_mini_cluster.cc:1428] Setting key b92a0e40ce4dfdd7c4cbe133a54eaf0b
I20260808 13:10:32.896637 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--builtin_ntp_servers=127.31.59.148:43399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 13:10:32.900558 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:32.894288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:38277 (local address 127.31.59.190:34719)
0808 13:10:32.894485 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:32.894491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:32.896107 (+ 1616us) server_negotiation.cc:407] Connection header received
0808 13:10:32.898405 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:32.898409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:32.898462 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:32.898541 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:32.899960 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:32.899964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:32.899967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:32.899982 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:32.900015 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:32.900026 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:32.900114 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:32.900295 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 13:10:32.900343 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:10:32.901799 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:32.894850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34719 (local address 127.31.59.130:38277)
0808 13:10:32.895941 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:32.895988 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:32.898125 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:32.898799 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:32.898810 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:32.898929 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:32.899615 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:32.899618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:32.899676 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:32.899793 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:32.900110 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:32.900120 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:32.900126 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:10:32.900383 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 13:10:32.900633 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":302,"threads_started":1}
I20260808 13:10:32.903049 4845 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34719
I20260808 13:10:32.903332 4845 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:32.903904 4845 heartbeater.cc:511] Master 127.31.59.190:34719 requested a full tablet report, sending...
I20260808 13:10:32.905203 4512 ts_manager.cc:194] Registered new tserver with Master: 0c9e2835272f477dac312e3a02188a39 (127.31.59.130:43197)
W20260808 13:10:33.222160 4848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:33.222440 4848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:33.222538 4848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:33.229671 4848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:33.229768 4848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:33.229880 4848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:33.238871 4848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:33.240119 4848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:33.241825 4848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:33.254323 4855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:33.254295 4853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:33.254299 4857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:33.255383 4848 server_base.cc:1034] running on GCE node
I20260808 13:10:33.255959 4848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:33.256853 4848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:33.258049 4848 hybrid_clock.cc:648] HybridClock initialized: now 1786194633257996 us; error 44 us; skew 500 ppm
I20260808 13:10:33.258349 4848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:33.260571 4848 webserver.cc:460] Webserver started at http://127.31.59.131:44137/ using document root <none> and password file <none>
I20260808 13:10:33.261225 4848 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:33.261333 4848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:33.261713 4848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:33.264467 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "84d67b9ac30743408b8298fc4e4c5053"
format_stamp: "Formatted at 2026-08-08 13:10:33 on dist-test-slave-49h3"
server_key: "61160ff7aee859ed5e11251ef82b96e3"
server_key_iv: "4a656fa0f2762adf6189e66592124714"
server_key_version: "encryptionkey@0"
I20260808 13:10:33.265211 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "84d67b9ac30743408b8298fc4e4c5053"
format_stamp: "Formatted at 2026-08-08 13:10:33 on dist-test-slave-49h3"
server_key: "61160ff7aee859ed5e11251ef82b96e3"
server_key_iv: "4a656fa0f2762adf6189e66592124714"
server_key_version: "encryptionkey@0"
I20260808 13:10:33.270211 4848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:10:33.273483 4864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:33.274755 4848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:33.274885 4848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "84d67b9ac30743408b8298fc4e4c5053"
format_stamp: "Formatted at 2026-08-08 13:10:33 on dist-test-slave-49h3"
server_key: "61160ff7aee859ed5e11251ef82b96e3"
server_key_iv: "4a656fa0f2762adf6189e66592124714"
server_key_version: "encryptionkey@0"
I20260808 13:10:33.275059 4848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:33.280342 4848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:33.281055 4848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:33.281281 4848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:33.282207 4848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:33.283489 4848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:33.283582 4848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:33.283667 4848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:33.283721 4848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:33.313361 4848 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:35159
I20260808 13:10:33.313405 4979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:35159 every 8 connection(s)
I20260808 13:10:33.314718 4848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:33.319320 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 4848
I20260808 13:10:33.319483 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:33.320091 31982 external_mini_cluster.cc:1428] Setting key 4b3c25dd84c273c7743b0f34d201bcc9
I20260808 13:10:33.323410 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.317198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:49351 (local address 127.31.59.190:34719)
0808 13:10:33.317411 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.317418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.318796 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 13:10:33.321114 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.321121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.321197 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.321303 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.322705 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.322711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.322716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.322741 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.322782 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.322799 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.322918 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.323114 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.323179 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:10:33.324368 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.317761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34719 (local address 127.31.59.131:49351)
0808 13:10:33.318635 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.318670 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.320795 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.321569 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.321581 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.321698 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.322415 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.322417 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.322461 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.322544 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.322886 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.322894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.322900 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.323162 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.323417 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":258,"threads_started":1}
I20260808 13:10:33.325654 4980 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34719
I20260808 13:10:33.325945 4980 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:33.326541 4980 heartbeater.cc:511] Master 127.31.59.190:34719 requested a full tablet report, sending...
I20260808 13:10:33.327845 4512 ts_manager.cc:194] Registered new tserver with Master: 84d67b9ac30743408b8298fc4e4c5053 (127.31.59.131:35159)
I20260808 13:10:33.335502 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:10:33.342229 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.339835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.31.59.190:34719)
0808 13:10:33.339988 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.339994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.340476 (+ 482us) server_negotiation.cc:407] Connection header received
0808 13:10:33.340632 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.340636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.340690 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.340776 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.341631 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.341637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.341641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.341658 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.341690 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.341701 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.341815 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.341903 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.341964 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:10:33.345849 4512 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:33.352730 4512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:33.370595 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.364650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:35159 (local address 127.0.0.1:58864)
0808 13:10:33.365558 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.365600 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.365739 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.368270 (+ 2531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.368279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.368377 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.369318 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.369322 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.369352 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.369432 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.369913 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.369926 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.369934 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.370086 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.370170 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"threads_started":1}
I20260808 13:10:33.372465 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.364725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34218)
0808 13:10:33.366813 (+ 2088us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.366830 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.366924 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.370910 (+ 3986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.370918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.370946 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.371633 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.371636 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.371655 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.371700 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.372065 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.372068 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.372072 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.372161 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.372199 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1955,"thread_start_us":435,"threads_started":1}
I20260808 13:10:33.372578 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.365347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.31.59.129:42185)
0808 13:10:33.370185 (+ 4838us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.370192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.370218 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:10:33.370293 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.370300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.370491 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.370638 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.371845 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.371852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.371865 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.371890 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.371944 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.371963 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.372086 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.372172 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.372285 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4687,"thread_start_us":4569,"threads_started":1}
I20260808 13:10:33.373776 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.364753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.31.59.131:35159)
0808 13:10:33.367849 (+ 3096us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.367855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.367882 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:10:33.367937 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.367950 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.368114 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.368225 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.369632 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.369642 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.369655 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.369675 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.369756 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.369779 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.373214 (+ 3435us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.373284 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.373376 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2973,"thread_start_us":2816,"threads_started":1}
I20260808 13:10:33.374359 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.366219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43197 (local address 127.0.0.1:41080)
0808 13:10:33.370805 (+ 4586us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.370821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.370901 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.372677 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.372682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.372703 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.373245 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.373249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.373264 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.373305 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.373947 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.373952 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.373957 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.374061 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.374108 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4476,"thread_start_us":346,"threads_started":1}
I20260808 13:10:33.374941 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.366796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.31.59.130:43197)
0808 13:10:33.367394 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.367405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.371015 (+ 3610us) server_negotiation.cc:407] Connection header received
0808 13:10:33.371092 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.371099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.371585 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.371743 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.373626 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.373634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.373646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.373675 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.373732 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.373765 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.373933 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.374552 (+ 619us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.374646 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":300,"threads_started":1}
I20260808 13:10:33.376384 4641 tablet_service.cc:1467] Processing CreateTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 (DEFAULT_TABLE table=test-table [id=04d0485eb22b45cf9ee2fd3bab812410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:33.377168 4913 tablet_service.cc:1467] Processing CreateTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 (DEFAULT_TABLE table=test-table [id=04d0485eb22b45cf9ee2fd3bab812410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:33.377951 4641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d67cef3137b479ebe90da6b58e80e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:33.378597 4913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d67cef3137b479ebe90da6b58e80e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:33.380437 4778 tablet_service.cc:1467] Processing CreateTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 (DEFAULT_TABLE table=test-table [id=04d0485eb22b45cf9ee2fd3bab812410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:33.381521 4778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d67cef3137b479ebe90da6b58e80e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:33.392594 4999 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Bootstrap starting.
I20260808 13:10:33.395977 4999 tablet_bootstrap.cc:654] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:33.396936 5000 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Bootstrap starting.
I20260808 13:10:33.398612 4999 log.cc:824] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:33.399693 5001 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Bootstrap starting.
I20260808 13:10:33.400710 5000 tablet_bootstrap.cc:654] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:33.402040 5000 log.cc:824] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:33.402520 5001 tablet_bootstrap.cc:654] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:33.403569 5001 log.cc:824] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:33.405563 5000 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: No bootstrap required, opened a new log
I20260808 13:10:33.405838 5000 ts_tablet_manager.cc:1397] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 13:10:33.405954 4999 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: No bootstrap required, opened a new log
I20260808 13:10:33.406221 4999 ts_tablet_manager.cc:1397] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.009s
I20260808 13:10:33.406262 5001 tablet_bootstrap.cc:492] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: No bootstrap required, opened a new log
I20260808 13:10:33.406482 5001 ts_tablet_manager.cc:1397] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 13:10:33.413112 5001 raft_consensus.cc:348] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.413486 5001 raft_consensus.cc:374] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:33.413584 5001 raft_consensus.cc:729] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d67b9ac30743408b8298fc4e4c5053, State: Initialized, Role: FOLLOWER
I20260808 13:10:33.414149 5001 consensus_queue.cc:260] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.415199 4980 heartbeater.cc:503] Master 127.31.59.190:34719 was elected leader, sending a full tablet report...
I20260808 13:10:33.415881 5001 ts_tablet_manager.cc:1428] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
I20260808 13:10:33.416181 4999 raft_consensus.cc:348] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.416181 5000 raft_consensus.cc:348] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.416615 5000 raft_consensus.cc:374] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:33.416615 4999 raft_consensus.cc:374] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:33.416707 5000 raft_consensus.cc:729] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7e419e83034ca1b76ee9d6ca21a2c6, State: Initialized, Role: FOLLOWER
I20260808 13:10:33.416707 4999 raft_consensus.cc:729] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9e2835272f477dac312e3a02188a39, State: Initialized, Role: FOLLOWER
I20260808 13:10:33.417338 5000 consensus_queue.cc:260] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.417330 4999 consensus_queue.cc:260] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.419023 4845 heartbeater.cc:503] Master 127.31.59.190:34719 was elected leader, sending a full tablet report...
I20260808 13:10:33.419271 5000 ts_tablet_manager.cc:1428] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 13:10:33.419579 4999 ts_tablet_manager.cc:1428] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260808 13:10:33.420127 4710 heartbeater.cc:503] Master 127.31.59.190:34719 was elected leader, sending a full tablet report...
W20260808 13:10:33.461472 4711 tablet.cc:2369] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:33.566331 4981 tablet.cc:2369] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:33.621557 5008 raft_consensus.cc:482] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:33.621897 5008 raft_consensus.cc:504] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.623680 5008 leader_election.cc:288] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d67b9ac30743408b8298fc4e4c5053 (127.31.59.131:35159), 0c9e2835272f477dac312e3a02188a39 (127.31.59.130:43197)
I20260808 13:10:33.627638 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.624131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:53641 (local address 127.31.59.130:43197)
0808 13:10:33.624325 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.624332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.625061 (+ 729us) server_negotiation.cc:407] Connection header received
0808 13:10:33.625662 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.625668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.625736 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.625844 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.627007 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.627013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.627018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.627039 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.627079 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.627094 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.627223 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.627303 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.627368 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:10:33.627635 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.624130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:41563 (local address 127.31.59.131:35159)
0808 13:10:33.624325 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.624332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.625059 (+ 727us) server_negotiation.cc:407] Connection header received
0808 13:10:33.625662 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.625668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.625736 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.625845 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.627007 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.627013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.627018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.627039 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.627079 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.627094 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.627223 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.627306 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.627367 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:10:33.627916 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.624084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:35159 (local address 127.31.59.129:41563)
0808 13:10:33.624944 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.624958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.625513 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.625891 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.625896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.625929 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.626773 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.626777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.626801 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.626855 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.627191 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.627197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.627202 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.627377 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.627438 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":16,"thread_start_us":434,"threads_started":1}
I20260808 13:10:33.628206 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.624034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43197 (local address 127.31.59.129:53641)
0808 13:10:33.624920 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.624944 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:33.625540 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:33.625891 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:33.625896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:33.625929 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:33.626833 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:10:33.626836 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:33.626847 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:10:33.626874 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:33.627191 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:33.627197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:33.627202 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:10:33.627352 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 13:10:33.627426 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"spinlock_wait_cycles":983936,"thread_start_us":507,"threads_started":1}
I20260808 13:10:33.628997 4798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d67cef3137b479ebe90da6b58e80e64" candidate_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9e2835272f477dac312e3a02188a39" is_pre_election: true
I20260808 13:10:33.628998 4933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d67cef3137b479ebe90da6b58e80e64" candidate_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d67b9ac30743408b8298fc4e4c5053" is_pre_election: true
I20260808 13:10:33.629487 4798 raft_consensus.cc:2454] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f7e419e83034ca1b76ee9d6ca21a2c6 in term 0.
I20260808 13:10:33.629493 4933 raft_consensus.cc:2454] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f7e419e83034ca1b76ee9d6ca21a2c6 in term 0.
I20260808 13:10:33.630128 4592 leader_election.cc:302] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9e2835272f477dac312e3a02188a39, 1f7e419e83034ca1b76ee9d6ca21a2c6; no voters:
I20260808 13:10:33.630507 5008 raft_consensus.cc:2781] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:33.630641 5008 raft_consensus.cc:482] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:33.630746 5008 raft_consensus.cc:3037] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:33.632925 5008 raft_consensus.cc:504] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.633570 5008 leader_election.cc:288] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [CANDIDATE]: Term 1 election: Requested vote from peers 84d67b9ac30743408b8298fc4e4c5053 (127.31.59.131:35159), 0c9e2835272f477dac312e3a02188a39 (127.31.59.130:43197)
I20260808 13:10:33.633999 4933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d67cef3137b479ebe90da6b58e80e64" candidate_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d67b9ac30743408b8298fc4e4c5053"
I20260808 13:10:33.634024 4798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d67cef3137b479ebe90da6b58e80e64" candidate_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9e2835272f477dac312e3a02188a39"
I20260808 13:10:33.634183 4933 raft_consensus.cc:3037] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:33.634191 4798 raft_consensus.cc:3037] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:33.636380 4933 raft_consensus.cc:2454] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f7e419e83034ca1b76ee9d6ca21a2c6 in term 1.
I20260808 13:10:33.636379 4798 raft_consensus.cc:2454] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f7e419e83034ca1b76ee9d6ca21a2c6 in term 1.
I20260808 13:10:33.636765 4592 leader_election.cc:302] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9e2835272f477dac312e3a02188a39, 1f7e419e83034ca1b76ee9d6ca21a2c6; no voters:
I20260808 13:10:33.637053 5008 raft_consensus.cc:2781] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:33.637519 5008 raft_consensus.cc:686] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1f7e419e83034ca1b76ee9d6ca21a2c6, State: Running, Role: LEADER
I20260808 13:10:33.638015 5008 consensus_queue.cc:237] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } }
I20260808 13:10:33.642549 4513 catalog_manager.cc:5495] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f7e419e83034ca1b76ee9d6ca21a2c6 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f7e419e83034ca1b76ee9d6ca21a2c6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 42185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 } health_report { overall_health: UNKNOWN } } }
W20260808 13:10:33.643563 4846 tablet.cc:2369] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:33.670112 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.668031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.31.59.129:42185)
0808 13:10:33.668221 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.668227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.668247 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:33.668437 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.668441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.668537 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.668623 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.669437 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.669500 (+ 63us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.669506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.669526 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.669560 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.669573 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.669672 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.669785 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.669845 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:10:33.678124 4933 raft_consensus.cc:1261] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 1 FOLLOWER]: Refusing update from remote peer 1f7e419e83034ca1b76ee9d6ca21a2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:33.678134 4798 raft_consensus.cc:1261] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 1 FOLLOWER]: Refusing update from remote peer 1f7e419e83034ca1b76ee9d6ca21a2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:33.679064 5011 consensus_queue.cc:1035] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d67b9ac30743408b8298fc4e4c5053" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:33.679545 5008 consensus_queue.cc:1035] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9e2835272f477dac312e3a02188a39" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:33.704804 5018 mvcc.cc:205] Tried to move back new op lower bound from 7316253219535302656 to 7316253219385040896. Current Snapshot: MvccSnapshot[applied={T|T < 7316253219535302656}]
I20260808 13:10:33.705058 4513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:33.705346 4513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:33.709209 4513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4a6d83b78b14bbdae7ca922dc05c2ad: Sending DeleteTablet for 3 replicas of tablet 5d67cef3137b479ebe90da6b58e80e64
I20260808 13:10:33.710429 4913 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:33 UTC) from {username='slave'} at 127.0.0.1:58864
I20260808 13:10:33.710517 4641 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:33 UTC) from {username='slave'} at 127.0.0.1:34218
I20260808 13:10:33.710736 4778 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d67cef3137b479ebe90da6b58e80e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:33 UTC) from {username='slave'} at 127.0.0.1:41080
I20260808 13:10:33.713011 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4576
I20260808 13:10:33.720414 5026 tablet_replica.cc:330] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: stopping tablet replica
I20260808 13:10:33.721956 5026 raft_consensus.cc:2229] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:33.722558 5026 raft_consensus.cc:2258] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:33.728304 5025 tablet_replica.cc:330] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: stopping tablet replica
I20260808 13:10:33.729502 5025 raft_consensus.cc:2229] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:33.730273 5025 raft_consensus.cc:2258] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:33.730899 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.715186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.31.59.129:42185)
0808 13:10:33.715380 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.715387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.727272 (+ 11885us) server_negotiation.cc:407] Connection header received
0808 13:10:33.727825 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.727832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.727909 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.727994 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.729973 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.729978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.729982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.730003 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.730042 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.730058 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.730165 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.730526 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.730598 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:10:33.732103 4704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:33.733469 5026 ts_tablet_manager.cc:1905] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:33.738571 5024 tablet_replica.cc:330] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: stopping tablet replica
I20260808 13:10:33.739089 5024 raft_consensus.cc:2229] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:33.739641 5024 raft_consensus.cc:2258] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:33.740073 5026 ts_tablet_manager.cc:1918] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:33.740250 5026 log.cc:1196] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/5d67cef3137b479ebe90da6b58e80e64
I20260808 13:10:33.740698 5026 ts_tablet_manager.cc:1939] T 5d67cef3137b479ebe90da6b58e80e64 P 0c9e2835272f477dac312e3a02188a39: Deleting consensus metadata
I20260808 13:10:33.742446 5024 ts_tablet_manager.cc:1905] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:33.742573 4496 catalog_manager.cc:4841] TS 0c9e2835272f477dac312e3a02188a39 (127.31.59.130:43197): tablet 5d67cef3137b479ebe90da6b58e80e64 (table test-table [id=04d0485eb22b45cf9ee2fd3bab812410]) successfully deleted
I20260808 13:10:33.747794 5024 ts_tablet_manager.cc:1918] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:33.747956 5024 log.cc:1196] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/5d67cef3137b479ebe90da6b58e80e64
I20260808 13:10:33.748369 5024 ts_tablet_manager.cc:1939] T 5d67cef3137b479ebe90da6b58e80e64 P 84d67b9ac30743408b8298fc4e4c5053: Deleting consensus metadata
I20260808 13:10:33.749877 4499 catalog_manager.cc:4841] TS 84d67b9ac30743408b8298fc4e4c5053 (127.31.59.131:35159): tablet 5d67cef3137b479ebe90da6b58e80e64 (table test-table [id=04d0485eb22b45cf9ee2fd3bab812410]) successfully deleted
I20260808 13:10:33.732609 5025 ts_tablet_manager.cc:1905] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:33.841261 5025 ts_tablet_manager.cc:1918] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:33.841496 5025 log.cc:1196] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/5d67cef3137b479ebe90da6b58e80e64
I20260808 13:10:33.842082 5025 ts_tablet_manager.cc:1939] T 5d67cef3137b479ebe90da6b58e80e64 P 1f7e419e83034ca1b76ee9d6ca21a2c6: Deleting consensus metadata
I20260808 13:10:33.842350 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4576
I20260808 13:10:33.863518 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4715
W20260808 13:10:33.863436 4496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:42185 (error 108)
I20260808 13:10:33.865126 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.864710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34256)
0808 13:10:33.864857 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.864968 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 13:10:33.865393 4496 catalog_manager.cc:4584] TS 1f7e419e83034ca1b76ee9d6ca21a2c6 (127.31.59.129:42185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d67cef3137b479ebe90da6b58e80e64: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
I20260808 13:10:33.872087 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.864434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.31.59.130:43197)
0808 13:10:33.864661 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.864665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.864684 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:10:33.865338 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.865343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.865477 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.865591 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.871501 (+ 5910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.871505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.871508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.871525 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.871561 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.871573 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.871681 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.871760 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.871810 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 13:10:33.872754 4840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:33.919338 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.918771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34268)
0808 13:10:33.919014 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.919139 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:10:33.920516 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.920116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34278)
0808 13:10:33.920279 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.920369 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:10:33.969942 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4715
I20260808 13:10:33.981083 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.980603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34288)
0808 13:10:33.980822 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.980920 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 13:10:33.982120 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.981741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34290)
0808 13:10:33.981898 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:33.981984 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 13:10:33.988587 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4848
I20260808 13:10:33.991218 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:33.989323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.31.59.131:35159)
0808 13:10:33.989535 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:33.989541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:33.989560 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:10:33.989705 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:33.989709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:33.989821 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:33.989899 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:33.990705 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:33.990710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:33.990713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:33.990728 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:33.990763 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:33.990774 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:33.990874 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:33.990908 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 13:10:33.990952 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:10:33.991856 4975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:34.062734 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:34.062160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34296)
0808 13:10:34.062420 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:34.062538 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:10:34.063815 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:34.063370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:42185 (local address 127.0.0.1:34306)
0808 13:10:34.063572 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:34.063661 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 13:10:34.079190 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4848
I20260808 13:10:34.096309 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 4480
I20260808 13:10:34.097496 4545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:34.206380 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 4480
2026-08-08T13:10:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2782 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 13:10:34.249258 31982 mini_oidc.cc:152] Starting JWKS server
I20260808 13:10:34.252147 31982 webserver.cc:460] Webserver started at https://127.0.0.1:36239/ using document root <none> and password file <none>
I20260808 13:10:34.252523 31982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:10:34.253134 31982 webserver.cc:460] Webserver started at http://127.0.0.1:36061/ using document root <none> and password file <none>
2026-08-08T13:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:34Z Disabled control of system clock
I20260808 13:10:34.285316 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33627
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:33627
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36239/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/testchainca.cert with env {}
W20260808 13:10:34.610481 5038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:34.610795 5038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:34.610944 5038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:34.618163 5038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:34.618296 5038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:34.618367 5038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:34.618451 5038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:34.618510 5038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:10:34.618597 5038 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:10:34.618649 5038 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36239/jwks.json
I20260808 13:10:34.628762 5038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:33627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36239/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:34.630154 5038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:34.631497 5038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:34.643175 5044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:34.643165 5043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:34.643185 5046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:34.643894 5038 server_base.cc:1034] running on GCE node
I20260808 13:10:34.644661 5038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:34.646078 5038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:34.647279 5038 hybrid_clock.cc:648] HybridClock initialized: now 1786194634647236 us; error 34 us; skew 500 ppm
I20260808 13:10:34.647609 5038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:34.649818 5038 webserver.cc:460] Webserver started at http://127.31.59.190:39737/ using document root <none> and password file <none>
I20260808 13:10:34.650489 5038 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:34.650590 5038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:34.650884 5038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:34.653837 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "917b6f58086847dbbb1462bc3d583f5f"
format_stamp: "Formatted at 2026-08-08 13:10:34 on dist-test-slave-49h3"
server_key: "8b8dc8c10c5c4e7d54bacaa534ca9dc5"
server_key_iv: "53707e6d30f5517d15232317ffaad501"
server_key_version: "encryptionkey@0"
I20260808 13:10:34.654649 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "917b6f58086847dbbb1462bc3d583f5f"
format_stamp: "Formatted at 2026-08-08 13:10:34 on dist-test-slave-49h3"
server_key: "8b8dc8c10c5c4e7d54bacaa534ca9dc5"
server_key_iv: "53707e6d30f5517d15232317ffaad501"
server_key_version: "encryptionkey@0"
I20260808 13:10:34.659749 5038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:10:34.663136 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:34.664428 5038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:10:34.664600 5038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "917b6f58086847dbbb1462bc3d583f5f"
format_stamp: "Formatted at 2026-08-08 13:10:34 on dist-test-slave-49h3"
server_key: "8b8dc8c10c5c4e7d54bacaa534ca9dc5"
server_key_iv: "53707e6d30f5517d15232317ffaad501"
server_key_version: "encryptionkey@0"
I20260808 13:10:34.664825 5038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:34.684221 5038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:34.685101 5038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:34.685415 5038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:34.703507 5038 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:33627
I20260808 13:10:34.703531 5107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:33627 every 8 connection(s)
I20260808 13:10:34.704877 5038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:34.708690 5108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:34.713493 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5038
I20260808 13:10:34.713725 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:34.714504 31982 external_mini_cluster.cc:1428] Setting key a1a7e2eb267664577e90e08f1ee0b7ef
I20260808 13:10:34.720227 5108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f: Bootstrap starting.
I20260808 13:10:34.723369 5108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:34.724416 5108 log.cc:824] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:34.727783 5108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f: No bootstrap required, opened a new log
I20260808 13:10:34.728494 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:34.716456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.31.59.190:33627)
0808 13:10:34.717519 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:34.717536 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:34.717580 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:10:34.719354 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:34.719375 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:34.719809 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:34.720141 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:34.720759 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.722094 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.723496 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.724180 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.724348 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:34.725629 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:34.725638 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:34.725656 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:34.726386 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:34.726489 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:34.726514 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:34.726639 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:34.726856 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 13:10:34.727193 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":396,"threads_started":1}
I20260808 13:10:34.732482 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:10:34.737548 5108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } }
I20260808 13:10:34.737913 5108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:34.737988 5108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917b6f58086847dbbb1462bc3d583f5f, State: Initialized, Role: FOLLOWER
I20260808 13:10:34.738471 5108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } }
I20260808 13:10:34.738668 5108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:34.738775 5108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:34.738902 5108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:34.741043 5108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } }
I20260808 13:10:34.741580 5108 leader_election.cc:302] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 917b6f58086847dbbb1462bc3d583f5f; no voters:
I20260808 13:10:34.742185 5108 leader_election.cc:288] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:34.742321 5114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:34.743029 5114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [term 1 LEADER]: Becoming Leader. State: Replica: 917b6f58086847dbbb1462bc3d583f5f, State: Running, Role: LEADER
I20260808 13:10:34.743536 5114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } }
I20260808 13:10:34.744184 5108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:34.746148 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 917b6f58086847dbbb1462bc3d583f5f. Latest consensus state: current_term: 1 leader_uuid: "917b6f58086847dbbb1462bc3d583f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } } }
I20260808 13:10:34.746544 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:34.747349 5115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "917b6f58086847dbbb1462bc3d583f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917b6f58086847dbbb1462bc3d583f5f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33627 } } }
I20260808 13:10:34.747740 5115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:34.748677 5119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:34.754212 5119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:34.763571 5119 catalog_manager.cc:1347] Generated new cluster ID: 501c8d2882ca4fe997ca00823282539e
I20260808 13:10:34.763693 5119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:34.792017 5119 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:34.793347 5119 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:34.802400 5119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 917b6f58086847dbbb1462bc3d583f5f: Generated new TSK 0
I20260808 13:10:34.803205 5119 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 13:10:34.852330 5141 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 13:10:34.852548 5141 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:10:34.853891 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:34.831086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.31.59.190:33627)
0808 13:10:34.831293 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:34.831299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:34.831752 (+ 453us) server_negotiation.cc:407] Connection header received
0808 13:10:34.832143 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:34.832150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:34.832247 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:34.832405 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:10:34.832915 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.834093 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.835546 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.836304 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.836484 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:34.836979 (+ 495us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:10:34.853118 (+ 16139us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 13:10:34.853534 (+ 416us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:10:34.854027 5112 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 13:10:34.855726 5140 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:34.830945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33627 (local address 127.0.0.1:60066)
0808 13:10:34.831652 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:34.831668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:34.831820 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:34.832422 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:34.832426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:34.832435 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:10:34.832773 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:34.832783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.834229 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.834233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:34.835391 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:34.835403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:34.836515 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:34.836519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:34.836645 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:34.836706 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:10:34.853480 (+ 16774us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 13:10:34.853813 (+ 333us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.190:33627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":385,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.59.190:33627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 13:10:34.857398 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5038
I20260808 13:10:34.858575 5103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:34.957438 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5038
2026-08-08T13:10:34Z chronyd exiting
I20260808 13:10:35.009825 31982 test_util.cc:175] -----------------------------------------------
I20260808 13:10:35.009972 31982 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194589892156-31982-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (765 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 13:10:35.013818 31982 mini_oidc.cc:152] Starting JWKS server
I20260808 13:10:35.015632 31982 webserver.cc:460] Webserver started at https://127.0.0.1:45761/ using document root <none> and password file <none>
I20260808 13:10:35.015820 31982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:10:35.016422 31982 webserver.cc:460] Webserver started at http://127.0.0.1:39221/ using document root <none> and password file <none>
2026-08-08T13:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:35Z Disabled control of system clock
I20260808 13:10:35.048424 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:43521
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:34959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:43521
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45761/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/testchainca.cert with env {}
W20260808 13:10:35.375059 5148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:35.375361 5148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:35.375497 5148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:35.382835 5148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:35.382958 5148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:35.383025 5148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:35.383112 5148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:35.383170 5148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:10:35.383256 5148 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:10:35.383304 5148 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45761/jwks.json
I20260808 13:10:35.393671 5148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:43521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45761/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:35.395005 5148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:35.396335 5148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:35.407850 5153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:35.407899 5157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:35.407927 5155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:35.408756 5148 server_base.cc:1034] running on GCE node
I20260808 13:10:35.409622 5148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:35.411041 5148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:35.412256 5148 hybrid_clock.cc:648] HybridClock initialized: now 1786194635412192 us; error 53 us; skew 500 ppm
I20260808 13:10:35.412616 5148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:35.414883 5148 webserver.cc:460] Webserver started at http://127.31.59.190:33991/ using document root <none> and password file <none>
I20260808 13:10:35.415597 5148 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:35.415696 5148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:35.416013 5148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:35.419027 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "c800869bd1374dadb86e0786effbddd2"
format_stamp: "Formatted at 2026-08-08 13:10:35 on dist-test-slave-49h3"
server_key: "e8bd23d4c2e2084748347c1250deedbd"
server_key_iv: "8d15386558ab5f1f015119a615864a8e"
server_key_version: "encryptionkey@0"
I20260808 13:10:35.419802 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "c800869bd1374dadb86e0786effbddd2"
format_stamp: "Formatted at 2026-08-08 13:10:35 on dist-test-slave-49h3"
server_key: "e8bd23d4c2e2084748347c1250deedbd"
server_key_iv: "8d15386558ab5f1f015119a615864a8e"
server_key_version: "encryptionkey@0"
I20260808 13:10:35.424834 5148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:10:35.428332 5163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:35.429700 5148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:35.429870 5148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "c800869bd1374dadb86e0786effbddd2"
format_stamp: "Formatted at 2026-08-08 13:10:35 on dist-test-slave-49h3"
server_key: "e8bd23d4c2e2084748347c1250deedbd"
server_key_iv: "8d15386558ab5f1f015119a615864a8e"
server_key_version: "encryptionkey@0"
I20260808 13:10:35.430073 5148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:35.448297 5148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:35.449163 5148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:35.449501 5148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:35.467607 5148 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:43521
I20260808 13:10:35.467613 5218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:43521 every 8 connection(s)
I20260808 13:10:35.468999 5148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:35.472651 5220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:35.476102 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5148
I20260808 13:10:35.476310 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:35.477162 31982 external_mini_cluster.cc:1428] Setting key c29709fee8c8226d621e56387af4c797
I20260808 13:10:35.485678 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2: Bootstrap starting.
I20260808 13:10:35.489332 5220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:35.490499 5220 log.cc:824] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:35.490654 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:35.479102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.31.59.190:43521)
0808 13:10:35.480068 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:35.480085 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:35.480126 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:10:35.481799 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:35.481822 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:35.482293 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:35.482615 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:35.483230 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.484505 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.486287 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.486984 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.487114 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:35.488364 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:35.488372 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:35.488385 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:35.488920 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:35.489020 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:35.489041 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:35.489125 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:35.489368 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 13:10:35.489684 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":401,"threads_started":1}
I20260808 13:10:35.493477 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2: No bootstrap required, opened a new log
I20260808 13:10:35.494323 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:10:35.501206 5220 raft_consensus.cc:348] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } }
I20260808 13:10:35.501641 5220 raft_consensus.cc:374] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:35.501742 5220 raft_consensus.cc:729] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c800869bd1374dadb86e0786effbddd2, State: Initialized, Role: FOLLOWER
I20260808 13:10:35.502354 5220 consensus_queue.cc:260] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } }
I20260808 13:10:35.502584 5220 raft_consensus.cc:388] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:35.502712 5220 raft_consensus.cc:482] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:35.502872 5220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:35.505085 5220 raft_consensus.cc:504] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } }
I20260808 13:10:35.505645 5220 leader_election.cc:302] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c800869bd1374dadb86e0786effbddd2; no voters:
I20260808 13:10:35.506211 5220 leader_election.cc:288] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:35.506367 5226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:35.507057 5226 raft_consensus.cc:686] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [term 1 LEADER]: Becoming Leader. State: Replica: c800869bd1374dadb86e0786effbddd2, State: Running, Role: LEADER
I20260808 13:10:35.507567 5226 consensus_queue.cc:237] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } }
I20260808 13:10:35.508261 5220 sys_catalog.cc:564] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:35.510329 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c800869bd1374dadb86e0786effbddd2. Latest consensus state: current_term: 1 leader_uuid: "c800869bd1374dadb86e0786effbddd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } } }
I20260808 13:10:35.510768 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:35.513265 5236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:35.513242 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c800869bd1374dadb86e0786effbddd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c800869bd1374dadb86e0786effbddd2" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43521 } } }
I20260808 13:10:35.513660 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:35.518577 5236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:35.527839 5236 catalog_manager.cc:1347] Generated new cluster ID: 3a7d2a04995847409d9373cc9c63c828
I20260808 13:10:35.527954 5236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:35.547206 5236 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:35.548801 5236 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:35.555423 5236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c800869bd1374dadb86e0786effbddd2: Generated new TSK 0
I20260808 13:10:35.556214 5236 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 13:10:35.588802 5253 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 13:10:35.588943 5253 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:10:35.590140 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:35.571038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.31.59.190:43521)
0808 13:10:35.571206 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:35.571211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:35.571604 (+ 393us) server_negotiation.cc:407] Connection header received
0808 13:10:35.571858 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:35.571862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:35.571933 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:35.572088 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:10:35.572552 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.573286 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.574693 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.575159 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.575275 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:35.575764 (+ 489us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:10:35.589403 (+ 13639us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 13:10:35.589840 (+ 437us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 13:10:35.590287 5224 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 13:10:35.590373 5252 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:35.570881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43521 (local address 127.0.0.1:52382)
0808 13:10:35.571500 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:35.571520 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:35.571664 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:35.572120 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:35.572124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:35.572137 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:10:35.572387 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:35.572397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.573431 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.573436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:35.574553 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:35.574565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:35.575335 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:35.575342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:35.575479 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:35.575512 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:10:35.589700 (+ 14188us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 13:10:35.589980 (+ 280us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.190:43521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":354,"threads_started":1}
I20260808 13:10:35.591634 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5148
I20260808 13:10:35.593039 5214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:35.687355 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5148
2026-08-08T13:10:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 13:10:35.744387 31982 mini_oidc.cc:152] Starting JWKS server
I20260808 13:10:35.746426 31982 webserver.cc:460] Webserver started at https://127.0.0.1:32957/ using document root <none> and password file <none>
I20260808 13:10:35.746685 31982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:10:35.747308 31982 webserver.cc:460] Webserver started at http://127.0.0.1:41849/ using document root <none> and password file <none>
2026-08-08T13:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:35Z Disabled control of system clock
I20260808 13:10:35.781958 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42277
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42277
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32957/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 13:10:36.118673 5260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:36.119001 5260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:36.119149 5260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:36.126641 5260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:36.126772 5260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:36.126856 5260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:36.126942 5260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:36.127005 5260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:10:36.127101 5260 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:10:36.127156 5260 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32957/jwks.json
I20260808 13:10:36.136744 5260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32957/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:36.138134 5260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:36.139925 5260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:36.151947 5266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:36.152027 5265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:36.152058 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:36.152905 5260 server_base.cc:1034] running on GCE node
I20260808 13:10:36.153780 5260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:36.155203 5260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:36.156435 5260 hybrid_clock.cc:648] HybridClock initialized: now 1786194636156364 us; error 48 us; skew 500 ppm
I20260808 13:10:36.156771 5260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:36.159056 5260 webserver.cc:460] Webserver started at http://127.31.59.190:46131/ using document root <none> and password file <none>
I20260808 13:10:36.159762 5260 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:36.159868 5260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:36.160188 5260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:36.163184 5260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "0dff54f9d9b744868a41e143e4e55de4"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "094755f5aa1f787bc52cb6a1ab885f77"
server_key_iv: "a9c29a0b3e822dd9ed5d6bce3742dbf8"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.163985 5260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "0dff54f9d9b744868a41e143e4e55de4"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "094755f5aa1f787bc52cb6a1ab885f77"
server_key_iv: "a9c29a0b3e822dd9ed5d6bce3742dbf8"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.169225 5260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:36.172691 5275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:36.174063 5260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:36.174228 5260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "0dff54f9d9b744868a41e143e4e55de4"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "094755f5aa1f787bc52cb6a1ab885f77"
server_key_iv: "a9c29a0b3e822dd9ed5d6bce3742dbf8"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.174448 5260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:36.194206 5260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:36.195106 5260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:36.195420 5260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:36.214025 5260 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:42277
I20260808 13:10:36.214037 5329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:42277 every 8 connection(s)
I20260808 13:10:36.215436 5260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:36.219630 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:36.223008 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5260
I20260808 13:10:36.223264 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:36.224064 31982 external_mini_cluster.cc:1428] Setting key 236d7fdf80355251ef069c8b81a2755d
I20260808 13:10:36.232944 5330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4: Bootstrap starting.
I20260808 13:10:36.236586 5330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:36.237764 5330 log.cc:824] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:36.238026 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:36.226227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.31.59.190:42277)
0808 13:10:36.227261 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:36.227272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:36.227316 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:10:36.228996 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:36.229019 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:36.229583 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:36.229926 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:36.230548 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.231817 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.233639 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.234316 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.234451 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:36.235679 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:36.235686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:36.235700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:36.236238 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:36.236340 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:36.236362 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:36.236444 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:36.236679 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 13:10:36.237010 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":372,"threads_started":1}
I20260808 13:10:36.240728 5330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4: No bootstrap required, opened a new log
I20260808 13:10:36.241628 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:10:36.249051 5330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } }
I20260808 13:10:36.249461 5330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:36.249562 5330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff54f9d9b744868a41e143e4e55de4, State: Initialized, Role: FOLLOWER
I20260808 13:10:36.250089 5330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } }
I20260808 13:10:36.250299 5330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:36.250432 5330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:36.250588 5330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:36.252782 5330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } }
I20260808 13:10:36.253285 5330 leader_election.cc:302] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dff54f9d9b744868a41e143e4e55de4; no voters:
I20260808 13:10:36.253942 5330 leader_election.cc:288] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:36.254099 5335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:36.254827 5335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [term 1 LEADER]: Becoming Leader. State: Replica: 0dff54f9d9b744868a41e143e4e55de4, State: Running, Role: LEADER
I20260808 13:10:36.255401 5335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } }
I20260808 13:10:36.256032 5330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:36.259879 5337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dff54f9d9b744868a41e143e4e55de4. Latest consensus state: current_term: 1 leader_uuid: "0dff54f9d9b744868a41e143e4e55de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } } }
I20260808 13:10:36.260275 5337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:36.260762 5336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dff54f9d9b744868a41e143e4e55de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff54f9d9b744868a41e143e4e55de4" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42277 } } }
I20260808 13:10:36.261176 5336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:36.262918 5343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:36.268244 5343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:36.277191 5343 catalog_manager.cc:1347] Generated new cluster ID: fcbaa5d271944071883de219b2ce3686
I20260808 13:10:36.277316 5343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:36.296633 5343 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:36.298314 5343 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:36.319203 5343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dff54f9d9b744868a41e143e4e55de4: Generated new TSK 0
I20260808 13:10:36.320406 5343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:10:36.353438 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:36.345173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.31.59.190:42277)
0808 13:10:36.345388 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:36.345394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:36.345959 (+ 565us) server_negotiation.cc:407] Connection header received
0808 13:10:36.346241 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:36.346246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:36.346323 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:36.346465 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:10:36.347027 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.347835 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.349285 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.349794 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.349925 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:36.350463 (+ 538us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:10:36.352778 (+ 2315us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32957/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 13:10:36.353128 (+ 350us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32957/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 13:10:36.353596 5333 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32957/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 13:10:36.355552 5359 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:36.345131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42277 (local address 127.0.0.1:41870)
0808 13:10:36.345849 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:36.345865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:36.345997 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:36.346509 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:36.346515 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:36.346527 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:10:36.346860 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:36.346871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.347994 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.347998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:36.349132 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:36.349146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.350016 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.350023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:36.350159 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:36.350198 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:10:36.353012 (+ 2814us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32957/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 13:10:36.355044 (+ 2032us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.190:42277: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32957/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":1681,"thread_start_us":401,"threads_started":1}
I20260808 13:10:36.356747 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5260
I20260808 13:10:36.357896 5325 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 13:10:36.368755 5360 webserver.cc:533] Webserver: sslize failed:
E20260808 13:10:36.368897 5360 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:10:36.456491 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5260
2026-08-08T13:10:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (768 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T13:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:36Z Disabled control of system clock
I20260808 13:10:36.546185 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40343
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40343
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/testchainca.cert with env {}
W20260808 13:10:36.879575 5366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:36.879894 5366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:36.879992 5366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:36.887396 5366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:36.887511 5366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:36.887557 5366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:36.887614 5366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:36.887665 5366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:10:36.887766 5366 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:10:36.887822 5366 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 13:10:36.898339 5366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:36.899641 5366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:36.900928 5366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:36.912582 5374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:36.912604 5372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:36.912998 5371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:36.913930 5366 server_base.cc:1034] running on GCE node
I20260808 13:10:36.914703 5366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:36.916095 5366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:36.917299 5366 hybrid_clock.cc:648] HybridClock initialized: now 1786194636917252 us; error 34 us; skew 500 ppm
I20260808 13:10:36.917670 5366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:10:36.919843 5366 webserver.cc:460] Webserver started at http://127.31.59.190:36455/ using document root <none> and password file <none>
I20260808 13:10:36.920517 5366 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:36.920624 5366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:36.920919 5366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:36.923866 5366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "b9de384109564c90a7a55f68bf2727da"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "f9f0304792ec08fc9ab301c2aaf9e09f"
server_key_iv: "5d3cce2b3cfb53a8edc2103786999e56"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.924649 5366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "b9de384109564c90a7a55f68bf2727da"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "f9f0304792ec08fc9ab301c2aaf9e09f"
server_key_iv: "5d3cce2b3cfb53a8edc2103786999e56"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.929911 5366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:10:36.933251 5380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:36.934569 5366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:36.934744 5366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "b9de384109564c90a7a55f68bf2727da"
format_stamp: "Formatted at 2026-08-08 13:10:36 on dist-test-slave-49h3"
server_key: "f9f0304792ec08fc9ab301c2aaf9e09f"
server_key_iv: "5d3cce2b3cfb53a8edc2103786999e56"
server_key_version: "encryptionkey@0"
I20260808 13:10:36.934988 5366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:36.949891 5366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:36.950775 5366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:36.951090 5366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:36.969434 5366 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40343
I20260808 13:10:36.969477 5434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40343 every 8 connection(s)
I20260808 13:10:36.970865 5366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:36.974589 5435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:36.979593 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5366
I20260808 13:10:36.979789 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:36.980432 31982 external_mini_cluster.cc:1428] Setting key d3da1a6db8c622d6b0992be880d3cab5
I20260808 13:10:36.985350 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da: Bootstrap starting.
I20260808 13:10:36.989125 5435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:36.990383 5435 log.cc:824] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:36.993793 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da: No bootstrap required, opened a new log
I20260808 13:10:36.994604 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:36.982298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.31.59.190:40343)
0808 13:10:36.983321 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:36.983332 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:36.983378 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:10:36.985079 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:36.985102 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:36.985630 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:36.985973 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:36.986669 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.987969 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.989781 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:36.990478 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:36.990668 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:36.991843 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:36.991850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:36.991864 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:10:36.992456 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:36.992576 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:10:36.992603 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:36.992718 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:36.993044 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 13:10:36.993600 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":368,"threads_started":1}
I20260808 13:10:36.998230 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:10:37.003191 5435 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } }
I20260808 13:10:37.003594 5435 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:37.003700 5435 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9de384109564c90a7a55f68bf2727da, State: Initialized, Role: FOLLOWER
I20260808 13:10:37.004333 5435 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } }
I20260808 13:10:37.004598 5435 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:37.004726 5435 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:37.004860 5435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:37.007088 5435 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } }
I20260808 13:10:37.007561 5435 leader_election.cc:302] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9de384109564c90a7a55f68bf2727da; no voters:
I20260808 13:10:37.008169 5435 leader_election.cc:288] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:37.008361 5440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:37.009166 5440 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [term 1 LEADER]: Becoming Leader. State: Replica: b9de384109564c90a7a55f68bf2727da, State: Running, Role: LEADER
I20260808 13:10:37.009724 5440 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } }
I20260808 13:10:37.010524 5435 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:37.012295 5441 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9de384109564c90a7a55f68bf2727da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } } }
I20260808 13:10:37.012409 5442 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9de384109564c90a7a55f68bf2727da. Latest consensus state: current_term: 1 leader_uuid: "b9de384109564c90a7a55f68bf2727da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9de384109564c90a7a55f68bf2727da" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40343 } } }
I20260808 13:10:37.012671 5441 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:37.012671 5442 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:37.014422 5447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:37.018635 5447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:37.027742 5447 catalog_manager.cc:1347] Generated new cluster ID: 25103adad73a4864b367189ba8b22b54
I20260808 13:10:37.027895 5447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:37.044937 5447 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:37.046846 5447 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:37.055405 5447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9de384109564c90a7a55f68bf2727da: Generated new TSK 0
I20260808 13:10:37.056229 5447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:10:37.171526 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:37.075171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.31.59.190:40343)
0808 13:10:37.075404 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:37.075411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:37.075793 (+ 382us) server_negotiation.cc:407] Connection header received
0808 13:10:37.076048 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:37.076053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:37.076128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:37.076262 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:10:37.076770 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:37.077610 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:37.079690 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:37.080114 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:37.080206 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:37.080769 (+ 563us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:10:37.170873 (+ 90104us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 13:10:37.171247 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55034: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 13:10:37.171684 5438 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55034: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 13:10:37.171958 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:37.074948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40343 (local address 127.0.0.1:55034)
0808 13:10:37.075684 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:37.075705 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:37.075863 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:37.076305 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:37.076310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:37.076323 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:10:37.076631 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:37.076643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:37.077780 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:37.077786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:37.079541 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:37.079556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:37.080287 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:37.080295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:37.080475 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:37.080522 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:10:37.171188 (+ 90666us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 13:10:37.171520 (+ 332us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.190:40343: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":402,"threads_started":1}
I20260808 13:10:37.173338 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5366
I20260808 13:10:37.174490 5430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:37.279035 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5366
2026-08-08T13:10:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (802 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (419 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:38 dist-test-slave-49h3 krb5kdc[5481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:38 dist-test-slave-49h3 krb5kdc[5481](info): set up 2 sockets
Aug 08 13:10:38 dist-test-slave-49h3 krb5kdc[5481](info): commencing operation
krb5kdc: starting...
W20260808 13:10:39.190018 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:39 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:10:39.422510 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39771
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:39771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:39.747479 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:39.747767 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:39.747867 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:39.755067 5495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:39.755177 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:39.755231 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:39.755297 5495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:39.755342 5495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:39.765303 5495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:39771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:39.766850 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:39.768183 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:39.779742 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:39.779706 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:39.779783 5503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:39.780184 5495 server_base.cc:1034] running on GCE node
I20260808 13:10:39.781068 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:39.782612 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:39.783826 5495 hybrid_clock.cc:648] HybridClock initialized: now 1786194639783787 us; error 36 us; skew 500 ppm
I20260808 13:10:39.784157 5495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:39 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194639, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:39.789858 5495 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:10:39.791306 5495 webserver.cc:460] Webserver started at http://127.31.59.190:44323/ using document root <none> and password file <none>
I20260808 13:10:39.791977 5495 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:39.792105 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:39.792419 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:39.795475 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "798d4cca37074beb9ad9f8aa9044c758"
format_stamp: "Formatted at 2026-08-08 13:10:39 on dist-test-slave-49h3"
server_key: "a9e2e81922b7e3cfcd365710ffb0e704"
server_key_iv: "aa14f22f330aa94e6965dcbbe4523e4e"
server_key_version: "encryptionkey@0"
I20260808 13:10:39.796274 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "798d4cca37074beb9ad9f8aa9044c758"
format_stamp: "Formatted at 2026-08-08 13:10:39 on dist-test-slave-49h3"
server_key: "a9e2e81922b7e3cfcd365710ffb0e704"
server_key_iv: "aa14f22f330aa94e6965dcbbe4523e4e"
server_key_version: "encryptionkey@0"
I20260808 13:10:39.801388 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:39.804677 5510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:39.805984 5495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:39.806149 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "798d4cca37074beb9ad9f8aa9044c758"
format_stamp: "Formatted at 2026-08-08 13:10:39 on dist-test-slave-49h3"
server_key: "a9e2e81922b7e3cfcd365710ffb0e704"
server_key_iv: "aa14f22f330aa94e6965dcbbe4523e4e"
server_key_version: "encryptionkey@0"
I20260808 13:10:39.806337 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:39.825908 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:39.830045 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:39.830339 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:39.859742 5495 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:39771
I20260808 13:10:39.859782 5563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:39771 every 8 connection(s)
I20260808 13:10:39.862262 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:39.865345 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5495
I20260808 13:10:39.865805 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:39.867501 31982 external_mini_cluster.cc:1428] Setting key 83c8c233089dc9e5e71c7d3ad59acd2e
I20260808 13:10:39.871543 5564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:10:39 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:39.896790 5564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758: Bootstrap starting.
I20260808 13:10:39.903077 5564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:39.905222 5564 log.cc:824] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:39.909322 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:39.871486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.31.59.190:39771)
0808 13:10:39.872800 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:39.872824 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:39.872900 (+ 76us) server_negotiation.cc:407] Connection header received
0808 13:10:39.876006 (+ 3106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:39.876047 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:39.876763 (+ 716us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:39.877371 (+ 608us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:39.880845 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:39.883167 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:39.886328 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:39.887556 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:39.887836 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:39.896656 (+ 8820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:39.896675 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:39.896702 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:39.896799 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:39.902111 (+ 5312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:39.903542 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:39.903560 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:39.903566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:39.903854 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:39.904567 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:39.904572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:39.904574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:39.905693 (+ 1119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:39.906125 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:39.907059 (+ 934us) server_negotiation.cc:299] Negotiation successful
0808 13:10:39.907793 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":562,"threads_started":1}
I20260808 13:10:39.909907 5564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758: No bootstrap required, opened a new log
I20260808 13:10:39.923058 5564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } }
I20260808 13:10:39.923643 5564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:39.923838 5564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 798d4cca37074beb9ad9f8aa9044c758, State: Initialized, Role: FOLLOWER
I20260808 13:10:39.924734 5564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } }
I20260808 13:10:39.925074 5564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:39.925227 5564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:39.925554 5564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:39.929801 5564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } }
I20260808 13:10:39.930619 5564 leader_election.cc:302] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 798d4cca37074beb9ad9f8aa9044c758; no voters:
I20260808 13:10:39.931905 5564 leader_election.cc:288] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:39.932426 5570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:39.933759 5570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [term 1 LEADER]: Becoming Leader. State: Replica: 798d4cca37074beb9ad9f8aa9044c758, State: Running, Role: LEADER
I20260808 13:10:39.934899 5570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } }
I20260808 13:10:39.936532 5564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:39.939666 5572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 798d4cca37074beb9ad9f8aa9044c758. Latest consensus state: current_term: 1 leader_uuid: "798d4cca37074beb9ad9f8aa9044c758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } } }
I20260808 13:10:39.940282 5572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:39.941023 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "798d4cca37074beb9ad9f8aa9044c758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798d4cca37074beb9ad9f8aa9044c758" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39771 } } }
I20260808 13:10:39.941534 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:39.943101 5575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:39.958091 5575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:39.976588 5575 catalog_manager.cc:1347] Generated new cluster ID: 125be69183c04e40842b648639e5556f
I20260808 13:10:39.976790 5575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:39.989849 5575 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:39.992408 5575 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:40.008721 5575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 798d4cca37074beb9ad9f8aa9044c758: Generated new TSK 0
I20260808 13:10:40.011111 5575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:10:40.094937 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:40.673643 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:40.674242 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:40.674608 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:40.687564 5593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:40.687686 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:40.687860 5593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:40.705966 5593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:40.708217 5593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:40.710464 5593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:40.732115 5598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:40.732033 5599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:40.732092 5601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:40.733922 5593 server_base.cc:1034] running on GCE node
I20260808 13:10:40.735185 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:40.736569 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:40.737798 5593 hybrid_clock.cc:648] HybridClock initialized: now 1786194640737737 us; error 59 us; skew 500 ppm
I20260808 13:10:40.738580 5593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:40 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194640, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:40.744210 5593 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:10:40.746855 5593 webserver.cc:460] Webserver started at http://127.31.59.129:46011/ using document root <none> and password file <none>
I20260808 13:10:40.748103 5593 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:40.748327 5593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:40.748867 5593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:40.754243 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "130afa4462f6438298099df5304183c4"
format_stamp: "Formatted at 2026-08-08 13:10:40 on dist-test-slave-49h3"
server_key: "f198e54b64eb01ed9fedbe9a6dc87ead"
server_key_iv: "a5c95a7f4efc7177c56050e87ebcaaba"
server_key_version: "encryptionkey@0"
I20260808 13:10:40.755681 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "130afa4462f6438298099df5304183c4"
format_stamp: "Formatted at 2026-08-08 13:10:40 on dist-test-slave-49h3"
server_key: "f198e54b64eb01ed9fedbe9a6dc87ead"
server_key_iv: "a5c95a7f4efc7177c56050e87ebcaaba"
server_key_version: "encryptionkey@0"
I20260808 13:10:40.765239 5593 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.003s
I20260808 13:10:40.771849 5608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:40.774334 5593 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 13:10:40.774606 5593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "130afa4462f6438298099df5304183c4"
format_stamp: "Formatted at 2026-08-08 13:10:40 on dist-test-slave-49h3"
server_key: "f198e54b64eb01ed9fedbe9a6dc87ead"
server_key_iv: "a5c95a7f4efc7177c56050e87ebcaaba"
server_key_version: "encryptionkey@0"
I20260808 13:10:40.775068 5593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:40.835090 5593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:40.842574 5593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:40.843101 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:40.844899 5593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:40.847204 5593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:40.847339 5593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:40.847615 5593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:40.847759 5593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:40.905802 5593 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45383
I20260808 13:10:40.905885 5724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45383 every 8 connection(s)
I20260808 13:10:40.907600 5593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:40.908957 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5593
I20260808 13:10:40.909395 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:40.911073 31982 external_mini_cluster.cc:1428] Setting key dbb2cf614ec12bc7b5c794b047e25487
Aug 08 13:10:40 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194640, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:40.944743 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:40.912568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:38595 (local address 127.31.59.190:39771)
0808 13:10:40.913636 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:40.913659 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:40.915081 (+ 1422us) server_negotiation.cc:407] Connection header received
0808 13:10:40.918824 (+ 3743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:40.918832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:40.918964 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:40.919167 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:40.921922 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:40.923661 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:40.926578 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:40.927599 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:40.927743 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:40.933734 (+ 5991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:40.933742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:40.933746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:40.933805 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:40.938806 (+ 5001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:40.940062 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:40.940077 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:40.940087 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:40.940407 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:40.941182 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:40.941187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:40.941189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:40.941718 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:40.941975 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:40.943288 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0808 13:10:40.943956 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":502,"threads_started":1}
I20260808 13:10:40.946873 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:40.913704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39771 (local address 127.31.59.129:38595)
0808 13:10:40.914806 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:40.914900 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:40.918258 (+ 3358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:40.919483 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:40.919504 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:40.920553 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:40.921646 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:40.921679 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:40.923900 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:40.923913 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:40.926349 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:40.926372 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:40.927880 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:40.927887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:40.928246 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:40.929287 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:40.929342 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:40.933238 (+ 3896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:40.939078 (+ 5840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:40.939089 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:40.939114 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:40.939806 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:40.940659 (+ 853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:40.940673 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:40.940675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:40.940996 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:40.941929 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:40.941951 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:40.942579 (+ 628us) client_negotiation.cc:769] Sending connection context
0808 13:10:40.943036 (+ 457us) client_negotiation.cc:240] Negotiation successful
0808 13:10:40.944386 (+ 1350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":508,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
I20260808 13:10:40.949538 5725 heartbeater.cc:347] Connected to a master server at 127.31.59.190:39771
I20260808 13:10:40.949879 5725 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:40.950832 5725 heartbeater.cc:511] Master 127.31.59.190:39771 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
I20260808 13:10:40.954749 5527 ts_manager.cc:194] Registered new tserver with Master: 130afa4462f6438298099df5304183c4 (127.31.59.129:45383)
I20260808 13:10:40.957199 5527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:38595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:10:40.991990 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:41.559573 5731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:41.560083 5731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:41.560246 5731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:41.573421 5731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:41.573662 5731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:41.573975 5731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:41.591361 5731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:41.593343 5731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:41.595839 5731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:41.618036 5738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:41.618149 5740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:41.618495 5737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:41.619066 5731 server_base.cc:1034] running on GCE node
I20260808 13:10:41.620225 5731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:41.621868 5731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:41.623216 5731 hybrid_clock.cc:648] HybridClock initialized: now 1786194641623120 us; error 83 us; skew 500 ppm
I20260808 13:10:41.623775 5731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:41 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194641, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:41.630481 5731 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:10:41.632928 5731 webserver.cc:460] Webserver started at http://127.31.59.130:43437/ using document root <none> and password file <none>
I20260808 13:10:41.634217 5731 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:41.634404 5731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:41.634878 5731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:41.640203 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "7e173919f6504fc6acdb6a9e65d0e05e"
format_stamp: "Formatted at 2026-08-08 13:10:41 on dist-test-slave-49h3"
server_key: "7f0da2972232f8a9cbfdad61632933f5"
server_key_iv: "a08e2fabfb1137ff1da588827191072a"
server_key_version: "encryptionkey@0"
I20260808 13:10:41.641587 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "7e173919f6504fc6acdb6a9e65d0e05e"
format_stamp: "Formatted at 2026-08-08 13:10:41 on dist-test-slave-49h3"
server_key: "7f0da2972232f8a9cbfdad61632933f5"
server_key_iv: "a08e2fabfb1137ff1da588827191072a"
server_key_version: "encryptionkey@0"
I20260808 13:10:41.650070 5731 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260808 13:10:41.655711 5748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:41.658022 5731 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:10:41.658309 5731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "7e173919f6504fc6acdb6a9e65d0e05e"
format_stamp: "Formatted at 2026-08-08 13:10:41 on dist-test-slave-49h3"
server_key: "7f0da2972232f8a9cbfdad61632933f5"
server_key_iv: "a08e2fabfb1137ff1da588827191072a"
server_key_version: "encryptionkey@0"
I20260808 13:10:41.658623 5731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:41.681375 5731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:41.687590 5731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:41.688004 5731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:41.689692 5731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:41.691730 5731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:41.691866 5731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:41.692035 5731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:41.692111 5731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:41.744035 5731 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:41453
I20260808 13:10:41.744110 5865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:41453 every 8 connection(s)
I20260808 13:10:41.746071 5731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:41.752431 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5731
I20260808 13:10:41.752712 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:41.753556 31982 external_mini_cluster.cc:1428] Setting key 552788bd0818d283e1d7874b490319df
Aug 08 13:10:41 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194641, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:41.798246 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:41.751092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:37443 (local address 127.31.59.190:39771)
0808 13:10:41.752053 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:41.752073 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:41.754472 (+ 2399us) server_negotiation.cc:407] Connection header received
0808 13:10:41.757926 (+ 3454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:41.757934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:41.758082 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:41.758309 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:41.761552 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:41.763354 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:41.768139 (+ 4785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:41.769436 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:41.769619 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:41.776294 (+ 6675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:41.776309 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:41.776313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:41.776368 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:41.780841 (+ 4473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:41.782234 (+ 1393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:41.782240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:41.782242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:41.782435 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:41.784141 (+ 1706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:41.784147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:41.784149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:41.784631 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:41.784806 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:41.794486 (+ 9680us) server_negotiation.cc:299] Negotiation successful
0808 13:10:41.796089 (+ 1603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":410,"threads_started":1}
I20260808 13:10:41.828759 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:41.752485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39771 (local address 127.31.59.130:37443)
0808 13:10:41.754044 (+ 1559us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:41.754153 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:41.757306 (+ 3153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:41.758726 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:41.758747 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:41.759833 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:41.761154 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:41.761194 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:41.763684 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:41.763700 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:41.767818 (+ 4118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:41.767855 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:41.769747 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:41.769754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:41.770364 (+ 610us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:41.771688 (+ 1324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:41.771740 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:41.775931 (+ 4191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:41.781110 (+ 5179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:41.781133 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:41.781177 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:41.782055 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:41.782585 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:41.782590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:41.782593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:41.783961 (+ 1368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:41.784835 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:41.784852 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:41.788776 (+ 3924us) client_negotiation.cc:769] Sending connection context
0808 13:10:41.794558 (+ 5782us) client_negotiation.cc:240] Negotiation successful
0808 13:10:41.805683 (+ 11125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":629,"threads_started":1}
I20260808 13:10:41.842351 5866 heartbeater.cc:347] Connected to a master server at 127.31.59.190:39771
I20260808 13:10:41.844055 5866 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
I20260808 13:10:41.847594 5866 heartbeater.cc:511] Master 127.31.59.190:39771 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
I20260808 13:10:41.851624 5527 ts_manager.cc:194] Registered new tserver with Master: 7e173919f6504fc6acdb6a9e65d0e05e (127.31.59.130:41453)
I20260808 13:10:41.853658 5527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:37443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
W20260808 13:10:41.911864 31982 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.31.59.131: real 0.154s user 0.005s sys 0.034s
I20260808 13:10:41.918663 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:10:41.962656 5725 heartbeater.cc:503] Master 127.31.59.190:39771 was elected leader, sending a full tablet report...
W20260808 13:10:42.687243 5873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:42.687718 5873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:42.687989 5873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:42.699651 5873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:42.699818 5873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:42.700093 5873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:42.714624 5873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:39771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:42.717127 5873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:42.719231 5873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:42.746131 5879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:42.747169 5881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:42.747593 5883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:42.749750 5873 server_base.cc:1034] running on GCE node
I20260808 13:10:42.753309 5873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:42.757670 5873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:42.759508 5873 hybrid_clock.cc:648] HybridClock initialized: now 1786194642759318 us; error 108 us; skew 500 ppm
I20260808 13:10:42.760246 5873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:42 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194642, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:42.766646 5873 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:10:42.769340 5873 webserver.cc:460] Webserver started at http://127.31.59.131:38253/ using document root <none> and password file <none>
I20260808 13:10:42.770764 5873 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:42.770993 5873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:42.771890 5873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:42.777894 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "84f59fd61f544ea28c719c2c050eb17f"
format_stamp: "Formatted at 2026-08-08 13:10:42 on dist-test-slave-49h3"
server_key: "ae94fa0e3b6b3af5377e0a8865b53dd5"
server_key_iv: "c18694c38b79ca92fbbe9b2ccb88df9d"
server_key_version: "encryptionkey@0"
I20260808 13:10:42.779682 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "84f59fd61f544ea28c719c2c050eb17f"
format_stamp: "Formatted at 2026-08-08 13:10:42 on dist-test-slave-49h3"
server_key: "ae94fa0e3b6b3af5377e0a8865b53dd5"
server_key_iv: "c18694c38b79ca92fbbe9b2ccb88df9d"
server_key_version: "encryptionkey@0"
I20260808 13:10:42.792008 5873 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.003s
I20260808 13:10:42.798218 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:42.800858 5873 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:10:42.801079 5873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "84f59fd61f544ea28c719c2c050eb17f"
format_stamp: "Formatted at 2026-08-08 13:10:42 on dist-test-slave-49h3"
server_key: "ae94fa0e3b6b3af5377e0a8865b53dd5"
server_key_iv: "c18694c38b79ca92fbbe9b2ccb88df9d"
server_key_version: "encryptionkey@0"
I20260808 13:10:42.801384 5873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:42.845325 5873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:42.853498 5873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:42.854305 5873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:42.857735 5873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:42.859700 5866 heartbeater.cc:503] Master 127.31.59.190:39771 was elected leader, sending a full tablet report...
I20260808 13:10:42.860865 5873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:42.860997 5873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.002s sys 0.000s
I20260808 13:10:42.861203 5873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:42.861300 5873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:43.061208 6011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:45295 every 8 connection(s)
I20260808 13:10:43.061276 5873 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:45295
I20260808 13:10:43.070218 5873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:43.074051 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 5873
I20260808 13:10:43.074438 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:43.075778 31982 external_mini_cluster.cc:1428] Setting key 84bed024114110df1d5420a24f9f17ff
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194642, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:10:43.100203 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.074946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:50799 (local address 127.31.59.190:39771)
0808 13:10:43.076645 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.076664 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.078455 (+ 1791us) server_negotiation.cc:407] Connection header received
0808 13:10:43.080968 (+ 2513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.080973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.081066 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.081220 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.084719 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.087751 (+ 3032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.089821 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.090287 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.090383 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.094249 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.094254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.094258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.094292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.097097 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.097941 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.097947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.097950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.098081 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.098501 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.098504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.098506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.098821 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.098931 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.099663 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.099857 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1444,"thread_start_us":923,"threads_started":1}
I20260808 13:10:43.101162 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.076789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39771 (local address 127.31.59.131:50799)
0808 13:10:43.078155 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:43.078257 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:43.080680 (+ 2423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:43.081570 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:43.081586 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:43.082556 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:43.084343 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:43.084434 (+ 91us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.088124 (+ 3690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.088130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:43.089671 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:43.089684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.090475 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.090480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:43.090690 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.091455 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:43.091478 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:43.094011 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:43.097222 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:43.097249 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:43.097304 (+ 55us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:43.097788 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:43.098170 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:43.098173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:43.098175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:43.098410 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:43.098919 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:43.098924 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:43.099305 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 13:10:43.099529 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 13:10:43.100001 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":471,"threads_started":1}
I20260808 13:10:43.102700 6012 heartbeater.cc:347] Connected to a master server at 127.31.59.190:39771
I20260808 13:10:43.102998 6012 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:43.103829 6012 heartbeater.cc:511] Master 127.31.59.190:39771 requested a full tablet report, sending...
I20260808 13:10:43.105367 5527 ts_manager.cc:194] Registered new tserver with Master: 84f59fd61f544ea28c719c2c050eb17f (127.31.59.131:45295)
I20260808 13:10:43.106257 5527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:50799
I20260808 13:10:43.117383 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/test-admin.keytab.
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.167449 31982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 13:10:43.178372 6020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:43.173584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.31.59.129:45383)
0808 13:10:43.174937 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.174944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.174975 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:10:43.175324 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.175345 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.175780 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.175917 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.177637 (+ 1720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42304: BlockingRecv error: recv got EOF from 127.0.0.1:42304 (error 108)
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":732,"threads_started":1}
W20260808 13:10:43.178956 31982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.181695 31982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.184171 31982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:43.210706 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.186934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.31.59.129:45383)
0808 13:10:43.187261 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.187271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.187490 (+ 219us) server_negotiation.cc:407] Connection header received
0808 13:10:43.187756 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.187760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.187848 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.187952 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.189045 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.190598 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.192921 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.194218 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.194413 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.200532 (+ 6119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.200549 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.200571 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.200630 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.205589 (+ 4959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.206571 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.206581 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.206583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.206821 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.207521 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.207525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.207526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.208537 (+ 1011us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.209002 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.209704 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.210182 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260808 13:10:43.222141 6020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:43.218650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.31.59.129:45383)
0808 13:10:43.218962 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.218974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.219597 (+ 623us) server_negotiation.cc:407] Connection header received
0808 13:10:43.219920 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.219930 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.220147 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.220346 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.221768 (+ 1422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42326: BlockingRecv error: recv got EOF from 127.0.0.1:42326 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 13:10:43.222340 31982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.224714 31982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.227087 31982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:43.247745 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.229050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.31.59.129:45383)
0808 13:10:43.229230 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.229234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.229770 (+ 536us) server_negotiation.cc:407] Connection header received
0808 13:10:43.229960 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.229963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.230057 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.230187 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.231688 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.232782 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.234655 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.235309 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.235469 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.240493 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.240500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.240503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.240547 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.244200 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.245097 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.245101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.245104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.245290 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.245804 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.245813 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.245815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.246209 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.246609 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.247045 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.247348 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 13:10:43.254552 6020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:43.251555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.31.59.129:45383)
0808 13:10:43.251828 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.251841 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.252484 (+ 643us) server_negotiation.cc:407] Connection header received
0808 13:10:43.252651 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.252655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.252759 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.253000 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.254029 (+ 1029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42340: BlockingRecv error: recv got EOF from 127.0.0.1:42340 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260808 13:10:43.254805 31982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194589892156-31982-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.257905 31982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:43.260367 31982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:10:43.289132 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.263219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.31.59.129:45383)
0808 13:10:43.263969 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.263989 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.264785 (+ 796us) server_negotiation.cc:407] Connection header received
0808 13:10:43.265121 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.265133 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.265248 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.265387 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.267736 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.268690 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.271264 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.272122 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.272390 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.279248 (+ 6858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.279253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.279256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.279292 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.285192 (+ 5900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.286429 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.286435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.286443 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.286697 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.287129 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.287132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.287134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.287621 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.287990 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.288316 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.288610 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260808 13:10:43.291926 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5593
I20260808 13:10:43.312335 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.293095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.31.59.129:45383)
0808 13:10:43.293325 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.293336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.294077 (+ 741us) server_negotiation.cc:407] Connection header received
0808 13:10:43.294318 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.294323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.294405 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.294517 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.296141 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.297318 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.299869 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.301173 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.301344 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.304222 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.304235 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.304245 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.304291 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.308249 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.309192 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.309196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.309199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.309393 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.309980 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.309985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.309987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.310558 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.310996 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.311405 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.311705 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:10:43.313207 5719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:43.440157 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5593
I20260808 13:10:43.487260 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5731
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:10:43.516741 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.488911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.31.59.130:41453)
0808 13:10:43.489795 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.489802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.489829 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:10:43.489893 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.489899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.490171 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.490393 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.491994 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.494240 (+ 2246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.496409 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.497704 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.497862 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.505717 (+ 7855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.505752 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.505778 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.505858 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.512035 (+ 6177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.512965 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.512973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.512976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.513182 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.513756 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.513764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.513772 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.514319 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.514781 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.515513 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.516162 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":472,"threads_started":1}
I20260808 13:10:43.519145 5861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:43.648932 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5731
I20260808 13:10:43.683827 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5873
Aug 08 13:10:43 dist-test-slave-49h3 krb5kdc[5481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:10:43.698515 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:43.684719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.31.59.131:45295)
0808 13:10:43.685401 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:43.685408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:43.685434 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:10:43.685540 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:43.685548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:43.685767 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:43.685904 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:43.686910 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.687915 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.689062 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:43.689810 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:43.689907 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:43.693531 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:43.693544 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:43.693558 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:43.693592 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:43.696151 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.696749 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.696755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.696757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.696889 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:43.697173 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:43.697176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:43.697178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:43.697551 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:43.697773 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:43.697999 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 13:10:43.698196 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":362,"threads_started":1}
I20260808 13:10:43.699805 6007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:43.783712 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5873
I20260808 13:10:43.803915 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 5495
I20260808 13:10:43.805107 5559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:43.920342 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 5495
2026-08-08T13:10:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (5817 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:44 dist-test-slave-49h3 krb5kdc[6043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:44 dist-test-slave-49h3 krb5kdc[6043](info): set up 2 sockets
Aug 08 13:10:44 dist-test-slave-49h3 krb5kdc[6043](info): commencing operation
krb5kdc: starting...
W20260808 13:10:45.049906 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:45 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab.
Entry for principal oryx/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab.
I20260808 13:10:45.298816 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34673
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab
--principal=oryx/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:34673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:45.650439 6056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:45.650758 6056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:45.650874 6056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:45.658349 6056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:45.658473 6056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:45.658547 6056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:45.658629 6056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:45.658686 6056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:45.668848 6056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:34673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab
--principal=oryx/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:45.670274 6056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:45.671633 6056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:45.683132 6062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:45.683094 6063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:45.683156 6066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:45.684126 6056 server_base.cc:1034] running on GCE node
I20260808 13:10:45.684880 6056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:45.686333 6056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:45.687575 6056 hybrid_clock.cc:648] HybridClock initialized: now 1786194645687523 us; error 38 us; skew 500 ppm
I20260808 13:10:45.687987 6056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:45 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:45.694795 6056 init.cc:377] Logged in from keytab as oryx/127.31.59.190@KRBTEST.COM (short username oryx)
I20260808 13:10:45.696333 6056 webserver.cc:460] Webserver started at http://127.31.59.190:45413/ using document root <none> and password file <none>
I20260808 13:10:45.697009 6056 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:45.697134 6056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:45.697428 6056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:45.700510 6056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "d82adb8c9a974543837b6ff639eae37c"
format_stamp: "Formatted at 2026-08-08 13:10:45 on dist-test-slave-49h3"
server_key: "5285c286933753cd6896b6f75405052e"
server_key_iv: "c451097c27743127cbdb2e33867cbb48"
server_key_version: "encryptionkey@0"
I20260808 13:10:45.701311 6056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "d82adb8c9a974543837b6ff639eae37c"
format_stamp: "Formatted at 2026-08-08 13:10:45 on dist-test-slave-49h3"
server_key: "5285c286933753cd6896b6f75405052e"
server_key_iv: "c451097c27743127cbdb2e33867cbb48"
server_key_version: "encryptionkey@0"
I20260808 13:10:45.706442 6056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:10:45.709733 6074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:45.710992 6056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:10:45.711153 6056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "d82adb8c9a974543837b6ff639eae37c"
format_stamp: "Formatted at 2026-08-08 13:10:45 on dist-test-slave-49h3"
server_key: "5285c286933753cd6896b6f75405052e"
server_key_iv: "c451097c27743127cbdb2e33867cbb48"
server_key_version: "encryptionkey@0"
I20260808 13:10:45.711330 6056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:45.739372 6056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:45.743535 6056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:45.743844 6056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:45.762966 6056 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:34673
I20260808 13:10:45.762984 6128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:34673 every 8 connection(s)
I20260808 13:10:45.764392 6056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:45.768676 6129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:45.773265 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6056
I20260808 13:10:45.773538 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:45.774171 31982 external_mini_cluster.cc:1428] Setting key 78afe8acb91d79e742bc9cdd7e2f2f04
I20260808 13:10:45.781172 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Bootstrap starting.
I20260808 13:10:45.784590 6129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:45.785777 6129 log.cc:824] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:10:45 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:45.788815 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: No bootstrap required, opened a new log
I20260808 13:10:45.797139 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:45.776208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.31.59.190:34673)
0808 13:10:45.777099 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:45.777114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:45.777165 (+ 51us) server_negotiation.cc:407] Connection header received
0808 13:10:45.779603 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:45.779626 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:45.780016 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:45.780351 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:45.781911 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:45.783320 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:45.784540 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:45.785320 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:45.785529 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:45.789836 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:45.789844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:45.789859 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:45.789898 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:45.792970 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:45.793715 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:45.793722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:45.793733 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:45.793914 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:45.794302 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:45.794306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:45.794308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:45.794886 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:45.795218 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:45.795657 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 13:10:45.796092 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":364,"threads_started":1}
I20260808 13:10:45.797472 6129 raft_consensus.cc:348] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } }
I20260808 13:10:45.797962 6129 raft_consensus.cc:374] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:45.798089 6129 raft_consensus.cc:729] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82adb8c9a974543837b6ff639eae37c, State: Initialized, Role: FOLLOWER
I20260808 13:10:45.798779 6129 consensus_queue.cc:260] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } }
I20260808 13:10:45.799027 6129 raft_consensus.cc:388] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:45.799165 6129 raft_consensus.cc:482] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:45.799342 6129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:45.801813 6129 raft_consensus.cc:504] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } }
I20260808 13:10:45.802294 6129 leader_election.cc:302] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d82adb8c9a974543837b6ff639eae37c; no voters:
I20260808 13:10:45.802878 6129 leader_election.cc:288] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:45.803040 6134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:45.804047 6134 raft_consensus.cc:686] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 LEADER]: Becoming Leader. State: Replica: d82adb8c9a974543837b6ff639eae37c, State: Running, Role: LEADER
I20260808 13:10:45.804555 6134 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } }
I20260808 13:10:45.805015 6129 sys_catalog.cc:564] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:45.807509 6135 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } }
I20260808 13:10:45.807565 6137 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82adb8c9a974543837b6ff639eae37c. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } }
I20260808 13:10:45.807941 6137 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:45.807941 6135 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:45.809052 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:45.814483 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:45.824316 6142 catalog_manager.cc:1347] Generated new cluster ID: 7a6fb6690c1d4b8abba1ec2d23a54360
I20260808 13:10:45.824442 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:45.852212 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:45.853578 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:45.866447 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Generated new TSK 0
I20260808 13:10:45.867326 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
I20260808 13:10:45.930474 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:46.272253 6156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:46.272539 6156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:46.272634 6156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:46.279891 6156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:46.280000 6156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:46.280126 6156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:46.290467 6156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:46.291728 6156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:46.293183 6156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:46.305889 6165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:46.305932 6163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:46.305984 6162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:46.307053 6156 server_base.cc:1034] running on GCE node
I20260808 13:10:46.307709 6156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:46.308622 6156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:46.309873 6156 hybrid_clock.cc:648] HybridClock initialized: now 1786194646309794 us; error 72 us; skew 500 ppm
I20260808 13:10:46.310160 6156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:46 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194646, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:46.313181 6156 init.cc:377] Logged in from keytab as oryx/127.31.59.129@KRBTEST.COM (short username oryx)
I20260808 13:10:46.314661 6156 webserver.cc:460] Webserver started at http://127.31.59.129:33219/ using document root <none> and password file <none>
I20260808 13:10:46.315305 6156 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:46.315407 6156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:46.315656 6156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:46.318742 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "c6f04941a99e4fe38adba3e29fbdbf18"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "5caafd6d34558593e2ef8060a9611576"
server_key_iv: "d8f06dc3a4f6df533c73bda0b16c9736"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.319531 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "c6f04941a99e4fe38adba3e29fbdbf18"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "5caafd6d34558593e2ef8060a9611576"
server_key_iv: "d8f06dc3a4f6df533c73bda0b16c9736"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.324574 6156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:10:46.328099 6172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.329408 6156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:46.329603 6156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "c6f04941a99e4fe38adba3e29fbdbf18"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "5caafd6d34558593e2ef8060a9611576"
server_key_iv: "d8f06dc3a4f6df533c73bda0b16c9736"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.329802 6156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:46.341297 6156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:46.345065 6156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:46.345352 6156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:46.346446 6156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:46.347903 6156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:46.347985 6156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.348116 6156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:46.348172 6156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.379812 6156 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44869
I20260808 13:10:46.379863 6294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44869 every 8 connection(s)
I20260808 13:10:46.381081 6156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:46.388257 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6156
I20260808 13:10:46.388499 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:46.389279 31982 external_mini_cluster.cc:1428] Setting key 7680d7471e7fafb9c8c5aa4a834b3f5c
Aug 08 13:10:46 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194646, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:46.401232 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:46.383568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:38621 (local address 127.31.59.190:34673)
0808 13:10:46.384025 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:46.384032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:46.384918 (+ 886us) server_negotiation.cc:407] Connection header received
0808 13:10:46.386714 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:46.386721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:46.386789 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:46.386874 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:46.388306 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.389407 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.391471 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.392042 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.392181 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:46.395723 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:46.395728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:46.395732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:46.395767 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:46.398197 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:46.399011 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:46.399014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:46.399016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:46.399127 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:46.399581 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:46.399584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:46.399586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:46.399848 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:46.399962 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:46.400790 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 13:10:46.400967 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":193,"threads_started":1}
I20260808 13:10:46.402662 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:46.384068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:38621)
0808 13:10:46.384766 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:46.384809 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:46.386462 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:46.387050 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:46.387057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:46.387550 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:46.388119 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:46.388136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.389606 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.389612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:46.391313 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:46.391335 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.392238 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.392243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:46.392433 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:46.393063 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:46.393084 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:46.395519 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:46.398337 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:46.398346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:46.398362 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:46.398913 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:46.399218 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:46.399222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:46.399226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:46.399475 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:46.399968 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:46.399975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:46.400398 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 13:10:46.400700 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 13:10:46.401224 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":265,"threads_started":1}
I20260808 13:10:46.404230 6295 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:46.404531 6295 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:46.405248 6295 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:46.407624 6092 ts_manager.cc:194] Registered new tserver with Master: c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:46.409139 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:38621
WARNING: no policy specified for oryx/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
I20260808 13:10:46.442912 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:46.774513 6303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:46.774902 6303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:46.775049 6303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:46.786218 6303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:46.786345 6303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:46.786468 6303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:46.796376 6303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:46.797719 6303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:46.799031 6303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:46.811882 6309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:46.811771 6312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:46.811802 6310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:46.812819 6303 server_base.cc:1034] running on GCE node
I20260808 13:10:46.813406 6303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:46.814395 6303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:46.815593 6303 hybrid_clock.cc:648] HybridClock initialized: now 1786194646815545 us; error 33 us; skew 500 ppm
I20260808 13:10:46.815914 6303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:46 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194646, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:46.819128 6303 init.cc:377] Logged in from keytab as oryx/127.31.59.130@KRBTEST.COM (short username oryx)
I20260808 13:10:46.820565 6303 webserver.cc:460] Webserver started at http://127.31.59.130:34675/ using document root <none> and password file <none>
I20260808 13:10:46.821225 6303 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:46.821345 6303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:46.821709 6303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:46.824698 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "591352facb7143999b896ae541d3f2c5"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "9dfa9637d823b9d2eae9a8a64cf41580"
server_key_iv: "cf185eb68c5eba46938d253b0a9dbc8f"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.825513 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "591352facb7143999b896ae541d3f2c5"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "9dfa9637d823b9d2eae9a8a64cf41580"
server_key_iv: "cf185eb68c5eba46938d253b0a9dbc8f"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.830710 6303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:10:46.834112 6320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.835433 6303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:46.835589 6303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "591352facb7143999b896ae541d3f2c5"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "9dfa9637d823b9d2eae9a8a64cf41580"
server_key_iv: "cf185eb68c5eba46938d253b0a9dbc8f"
server_key_version: "encryptionkey@0"
I20260808 13:10:46.835783 6303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:46.851043 6303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:46.854990 6303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:46.855274 6303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:46.856235 6303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:46.857601 6303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:46.857692 6303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.857821 6303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:46.857880 6303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:46.888670 6303 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:43315
I20260808 13:10:46.888700 6439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:43315 every 8 connection(s)
I20260808 13:10:46.890065 6303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:46.896108 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6303
I20260808 13:10:46.896318 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:46.896760 31982 external_mini_cluster.cc:1428] Setting key b7d0bc1df20993f8c0c3828c66de3faa
Aug 08 13:10:46 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194646, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:46.910868 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:46.892602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:56145 (local address 127.31.59.190:34673)
0808 13:10:46.892797 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:46.892803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:46.893974 (+ 1171us) server_negotiation.cc:407] Connection header received
0808 13:10:46.895702 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:46.895707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:46.895800 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:46.895910 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:46.897423 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.898406 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.900518 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.900955 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.901051 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:46.905090 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:46.905095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:46.905098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:46.905132 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:46.907689 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:46.908492 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:46.908496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:46.908498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:46.908617 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:46.909081 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:46.909086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:46.909089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:46.909478 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:46.909628 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:46.910299 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 13:10:46.910533 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:10:46.911723 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:46.893096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:56145)
0808 13:10:46.893813 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:46.893852 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:46.895454 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:46.896092 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:46.896100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:46.896648 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:46.897239 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:46.897265 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.898552 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.898556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:46.900370 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:46.900386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:46.901208 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:46.901215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:46.901525 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:46.902231 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:46.902259 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:46.904861 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:46.907829 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:46.907838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:46.907855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:46.908398 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:46.908707 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:46.908711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:46.908714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:46.908962 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:46.909590 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:46.909596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:46.909943 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 13:10:46.910179 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 13:10:46.910602 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":302,"threads_started":1}
I20260808 13:10:46.913107 6440 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:46.913421 6440 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:46.914165 6440 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:46.915614 6092 ts_manager.cc:194] Registered new tserver with Master: 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
I20260808 13:10:46.916437 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:56145
WARNING: no policy specified for oryx/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
I20260808 13:10:46.950824 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:47.281278 6445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:47.281581 6445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:47.281721 6445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:47.289036 6445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:47.289165 6445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:47.289321 6445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:47.299191 6445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:47.300522 6445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:47.301899 6445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:47.315307 6451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:47.315274 6454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:47.315263 6450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:47.316087 6445 server_base.cc:1034] running on GCE node
I20260808 13:10:47.316671 6445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:47.317678 6445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:47.318868 6445 hybrid_clock.cc:648] HybridClock initialized: now 1786194647318813 us; error 42 us; skew 500 ppm
I20260808 13:10:47.319203 6445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194647, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:47.322232 6445 init.cc:377] Logged in from keytab as oryx/127.31.59.131@KRBTEST.COM (short username oryx)
I20260808 13:10:47.323565 6445 webserver.cc:460] Webserver started at http://127.31.59.131:45167/ using document root <none> and password file <none>
I20260808 13:10:47.324219 6445 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:47.324333 6445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:47.324671 6445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:47.327641 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "8c7dd9c99c574d0489816e0505962ff8"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "e192953a6b198d3a8d840e8b2619887b"
server_key_iv: "99a47f76a734739a1dbe51bcbb5af293"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.328470 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "8c7dd9c99c574d0489816e0505962ff8"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "e192953a6b198d3a8d840e8b2619887b"
server_key_iv: "99a47f76a734739a1dbe51bcbb5af293"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.333825 6445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 13:10:47.337273 6461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:47.338646 6445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:47.338836 6445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8c7dd9c99c574d0489816e0505962ff8"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "e192953a6b198d3a8d840e8b2619887b"
server_key_iv: "99a47f76a734739a1dbe51bcbb5af293"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.339063 6445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:47.375362 6445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:47.379354 6445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:47.379640 6445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:47.380738 6445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:47.382198 6445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:47.382280 6445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:47.382417 6445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:47.382472 6445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:47.412070 6295 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
I20260808 13:10:47.413761 6445 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:45791
I20260808 13:10:47.413844 6584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:45791 every 8 connection(s)
I20260808 13:10:47.415189 6445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:47.415956 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6445
I20260808 13:10:47.416164 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:47.416698 31982 external_mini_cluster.cc:1428] Setting key cbb8bf104133a710a7ae24a10c33a251
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194647, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:47.434478 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.417886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:56747 (local address 127.31.59.190:34673)
0808 13:10:47.418347 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:47.418352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:47.419534 (+ 1182us) server_negotiation.cc:407] Connection header received
0808 13:10:47.421357 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:47.421362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:47.421427 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:47.421549 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:47.422922 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.423676 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.425093 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.425534 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.425631 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.428834 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:47.428839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:47.428842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:47.428874 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:47.431602 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.432411 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.432415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.432417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.432536 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.432919 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.432923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.432926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.433202 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:47.433323 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:47.434020 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 13:10:47.434202 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":233,"threads_started":1}
I20260808 13:10:47.435489 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.418541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:56747)
0808 13:10:47.419356 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:47.419415 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:47.421107 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:47.421724 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:47.421739 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:47.422240 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:47.422797 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.422814 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.423792 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.423796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.424984 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.424997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.425724 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.425740 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.425940 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.426532 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:47.426558 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:47.428648 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:47.431722 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.431745 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.431761 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.432290 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.432618 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.432620 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.432623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.432829 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.433324 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:47.433329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:47.433704 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 13:10:47.433952 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 13:10:47.434400 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":307,"threads_started":1}
I20260808 13:10:47.436681 6585 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:47.436966 6585 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:47.437747 6585 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:47.439131 6092 ts_manager.cc:194] Registered new tserver with Master: 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791)
I20260808 13:10:47.439953 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:56747
I20260808 13:10:47.444372 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.59.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.189.keytab.
Entry for principal oryx/127.31.59.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.189.keytab.
I20260808 13:10:47.498911 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:33935
--webserver_interface=127.31.59.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.189.keytab
--principal=oryx/127.31.59.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:34673,127.31.59.189:33935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:47.836083 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:47.836382 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:47.836514 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:47.844250 6591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:47.844370 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:47.844442 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:47.844512 6591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:47.844558 6591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:47.855113 6591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:34673,127.31.59.189:33935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.189.keytab
--principal=oryx/127.31.59.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:33935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
W20260808 13:10:47.855516 6591 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.59.190:34673,127.31.59.189:33935'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 13:10:47.856627 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:47.858019 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:47.869804 6596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:47.869871 6598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:47.869841 6600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:47.870842 6591 server_base.cc:1034] running on GCE node
I20260808 13:10:47.871696 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:47.873279 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:47.874516 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786194647874436 us; error 46 us; skew 500 ppm
I20260808 13:10:47.874871 6591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194647, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:47.878051 6591 init.cc:377] Logged in from keytab as oryx/127.31.59.189@KRBTEST.COM (short username oryx)
I20260808 13:10:47.879554 6591 webserver.cc:460] Webserver started at http://127.31.59.189:37853/ using document root <none> and password file <none>
I20260808 13:10:47.880251 6591 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:47.880353 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:47.880657 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:47.883957 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/instance:
uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "eb4b655efadb448a782633c0f821435c"
server_key_iv: "ef8a8c88ed4b2c4a967913d3c771170d"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.884750 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal/instance:
uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "eb4b655efadb448a782633c0f821435c"
server_key_iv: "ef8a8c88ed4b2c4a967913d3c771170d"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.890234 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 13:10:47.893803 6608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:47.895185 6591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:10:47.895349 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "eb4b655efadb448a782633c0f821435c"
server_key_iv: "ef8a8c88ed4b2c4a967913d3c771170d"
server_key_version: "encryptionkey@0"
I20260808 13:10:47.895558 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:47.915138 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:47.918946 6440 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
I20260808 13:10:47.919457 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:47.919719 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:47.941345 6591 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.189:33935
I20260808 13:10:47.941373 6664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.189:33935 every 8 connection(s)
I20260808 13:10:47.943197 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
I20260808 13:10:47.944141 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6591
I20260808 13:10:47.948025 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:47.957387 6665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194647, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.189@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:47.970608 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.946902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.31.59.189:33935)
0808 13:10:47.947971 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:47.947991 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:47.948428 (+ 437us) server_negotiation.cc:407] Connection header received
0808 13:10:47.950884 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:47.950916 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:47.951494 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:47.952011 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:47.953546 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.955596 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.956977 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.958058 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.958271 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.963164 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:47.963172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:47.963182 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:47.963217 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:47.966145 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.966877 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.966887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.966894 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.967096 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.967662 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.967666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.967668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.968243 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:47.968513 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:47.969054 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 13:10:47.969550 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":448,"threads_started":1}
I20260808 13:10:47.975545 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.960367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.31.59.190:34673)
0808 13:10:47.960865 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:47.960869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:47.962177 (+ 1308us) server_negotiation.cc:407] Connection header received
0808 13:10:47.962373 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:47.962379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:47.962472 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:47.962587 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:47.963749 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.964507 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.966007 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.966434 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.966557 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.970172 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:47.970178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:47.970181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:47.970222 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:47.972807 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.973692 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.973696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.973698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.973829 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.974215 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.974219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.974222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.974590 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:47.974729 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:47.975114 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 13:10:47.975299 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":242,"threads_started":1}
I20260808 13:10:47.975548 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.960412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54668)
0808 13:10:47.962010 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:47.962057 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:47.962224 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:47.962635 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:47.962640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:47.963292 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:47.963583 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.963593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.964658 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.964662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.965854 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.965867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.966621 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.966627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.966816 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.967438 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:47.967461 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:47.969962 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:47.972946 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.972956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.972961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.973554 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.973926 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.973929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.973932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.974063 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.974702 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:47.974708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:47.974862 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 13:10:47.975011 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 13:10:47.975261 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"thread_start_us":1213,"threads_started":1}
I20260808 13:10:47.977375 6665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:10:47 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194647, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.189@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
I20260808 13:10:47.989658 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.978199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55022)
0808 13:10:47.978367 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:47.978383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:47.978466 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:47.978744 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:47.978747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:47.979076 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:47.979211 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.979219 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.980250 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.980253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.981098 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:47.981109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.981891 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.981896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:47.982038 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.982544 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:47.982561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:47.984511 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:47.987529 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.987532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.987535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.987889 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.988215 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:47.988218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:47.988221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:47.988349 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:47.988872 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:47.988875 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:47.989002 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 13:10:47.989108 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:10:47.989355 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:10:47.989918 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:47.978372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.31.59.189:33935)
0808 13:10:47.978520 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:47.978525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:47.978539 (+ 14us) server_negotiation.cc:407] Connection header received
0808 13:10:47.978578 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:47.978581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:47.978644 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:47.978719 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:47.979374 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.980111 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.981235 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:47.981744 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:47.981836 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:47.984715 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:47.984720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:47.984723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:47.984753 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:47.987405 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.987982 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.987986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.987988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.988126 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:47.988464 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:47.988468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:47.988470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:47.988775 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:47.988893 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:47.989249 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 13:10:47.989535 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":60}
I20260808 13:10:47.994838 6665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Bootstrap starting.
I20260808 13:10:47.997936 6665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:47.998982 6665 log.cc:824] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:48.001788 6665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: No bootstrap required, opened a new log
I20260808 13:10:48.008849 6665 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } }
I20260808 13:10:48.009164 6665 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:48.009248 6665 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86d96cbfd324ad1ac8d9c4acfae8471, State: Initialized, Role: FOLLOWER
I20260808 13:10:48.009802 6665 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } }
I20260808 13:10:48.010607 6672 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } } }
I20260808 13:10:48.010902 6672 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:48.011559 6665 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 13:10:48.020491 6683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:10:48.020676 6683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:10:48.023530 6591 master_runner.cc:428] Detected that this master c86d96cbfd324ad1ac8d9c4acfae8471 is joining an existing cluster
I20260808 13:10:48.023638 6591 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.59.189:33935
I20260808 13:10:48.044842 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.028399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.31.59.190:34673)
0808 13:10:48.028781 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.028787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.029211 (+ 424us) server_negotiation.cc:407] Connection header received
0808 13:10:48.029534 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.029541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.029642 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.029717 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.031313 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.032417 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.034505 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.035198 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.035302 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.037925 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.037932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.037937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.037979 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.041495 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.042312 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.042317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.042321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.042487 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.042991 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.042995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.042999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.043368 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.043504 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.044231 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.044474 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260808 13:10:48.046322 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.028404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54672)
0808 13:10:48.029094 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.029114 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.029330 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.029962 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.029967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.030714 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.031127 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.031141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.032608 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.032617 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.034330 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.034344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.035631 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.035638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.035819 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.036522 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.036550 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.037703 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.041644 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.041649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.041653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.042162 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.042639 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.042644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.042651 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.042838 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.043639 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.043644 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.043873 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.045410 (+ 1537us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.045811 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":384,"threads_started":1}
I20260808 13:10:48.053828 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.029724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55036 (local address 127.31.59.189:33935)
0808 13:10:48.029914 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.029918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.038566 (+ 8648us) server_negotiation.cc:407] Connection header received
0808 13:10:48.039041 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.039046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.039122 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.039180 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.040774 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.041725 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.043579 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.044246 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.044328 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.046555 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.046561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.046564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.046600 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.050537 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.051452 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.051456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.051460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.051620 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.052125 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.052129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.052132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.052583 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.052727 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.053243 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.053477 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:10:48.055222 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.029267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55036)
0808 13:10:48.038454 (+ 9187us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.038474 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.038921 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.039338 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.039342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.040242 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.040634 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.040646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.041916 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.041920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.043457 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.043470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.044589 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.044594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.044735 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.045290 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.045310 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.046385 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.050695 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.050699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.050707 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.051336 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.051752 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.051755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.051758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.052003 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.052816 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.052820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.052963 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.054448 (+ 1485us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.054761 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9056,"thread_start_us":8886,"threads_started":1}
WARNING: no policy specified for oryx/127.31.59.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.188@KRBTEST.COM" created.
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.190@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
I20260808 13:10:48.074786 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.054255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.31.59.189:33935)
0808 13:10:48.054422 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.054428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.057802 (+ 3374us) server_negotiation.cc:407] Connection header received
0808 13:10:48.058074 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.058078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.058158 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.058235 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.059370 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.060287 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.062045 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.063077 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.063244 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.068792 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.068797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.068800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.068831 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.071524 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.072310 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.072315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.072319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.072486 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.072923 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.072928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.072931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.073326 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.073497 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.074151 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.074402 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"thread_start_us":1922,"threads_started":1}
I20260808 13:10:48.075838 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.053777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55042)
0808 13:10:48.057673 (+ 3896us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.057716 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.057885 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.058318 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.058326 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.058986 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.059222 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.059233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.060423 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.060427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.061771 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.061815 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.063724 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.063733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.064026 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.064858 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.064881 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.068605 (+ 3724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.071680 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.071693 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.071701 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.072154 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.072634 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.072639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.072642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.072801 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.073619 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.073628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.073826 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.075093 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.075458 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3675,"thread_start_us":3515,"threads_started":1}
I20260808 13:10:48.078413 6092 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.59.189:33935: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
member_type: NON_VOTER
last_known_addr {
host: "127.31.59.189"
port: 33935
}
attrs {
promote: true
}
}
dest_uuid: "d82adb8c9a974543837b6ff639eae37c"
cas_config_opid_index: -1
I20260808 13:10:48.079650 6092 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } }
I20260808 13:10:48.088946 6696 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } }
I20260808 13:10:48.090843 6637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:48.092489 6696 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.59.189:33935
I20260808 13:10:48.093258 6697 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } }
I20260808 13:10:48.093612 6697 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:48.093617 6591 master.cc:537] Master@127.31.59.189:33935 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.188.keytab.
Entry for principal oryx/127.31.59.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.188.keytab.
I20260808 13:10:48.096575 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:35963
--webserver_interface=127.31.59.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.188.keytab
--principal=oryx/127.31.59.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:10:48.096843 6637 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 13:10:48.098119 6696 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:10:48.101722 6075 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 13:10:48.106963 6075 consensus_peers.cc:487] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c -> Peer c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189:33935): Couldn't send request to peer c86d96cbfd324ad1ac8d9c4acfae8471. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:10:48.108414 6591 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:48.108865 6591 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:48.109057 6591 tablet_replica.cc:330] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: stopping tablet replica
I20260808 13:10:48.126106 6591 master.cc:552] Master@127.31.59.189:33935 shutdown complete.
I20260808 13:10:48.126274 6591 master_runner.cc:305] Clearing existing system tablet
I20260808 13:10:48.126696 6591 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:48.126797 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:48.129040 6591 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 13:10:48.130534 6703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:48.130869 6591 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:48.131139 6591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 13:10:48.131227 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "eb4b655efadb448a782633c0f821435c"
server_key_iv: "ef8a8c88ed4b2c4a967913d3c771170d"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.135078 6591 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:48.139744 6591 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 13:10:48.139920 6591 log.cc:1196] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 13:10:48.140359 6591 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Deleting consensus metadata
I20260808 13:10:48.140590 6591 master_runner.cc:315] Copying system tablet from 127.31.59.190:34673
I20260808 13:10:48.142992 6591 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: tablet copy: Beginning tablet copy session from remote peer at address 127.31.59.190:34673
I20260808 13:10:48.161082 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.143804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.31.59.190:34673)
0808 13:10:48.144008 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.144014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.150153 (+ 6139us) server_negotiation.cc:407] Connection header received
0808 13:10:48.150332 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.150338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.150435 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.150525 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.151462 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.152199 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.153436 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.153944 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.154063 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.155933 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.155938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.155941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.155974 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.158512 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.159161 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.159167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.159171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.159320 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.159683 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.159687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.159689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.159968 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.160109 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.160501 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.160712 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:10:48.161152 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.143700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54680)
0808 13:10:48.149903 (+ 6203us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.149926 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.150200 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.150574 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.150579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.151035 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.151297 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.151307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.152366 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.152370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.153292 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.153304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.154097 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.154103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.154278 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.154892 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.154914 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.155723 (+ 809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.158638 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.158644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.158649 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.159044 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.159428 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.159434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.159438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.159581 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.160090 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.160095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.160248 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.160370 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.160706 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6059,"thread_start_us":5938,"threads_started":1}
I20260808 13:10:48.162434 6113 tablet_copy_service.cc:140] P d82adb8c9a974543837b6ff639eae37c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c86d96cbfd324ad1ac8d9c4acfae8471 ({username='oryx', principal='oryx/127.31.59.189@KRBTEST.COM'} at 127.0.0.1:54680)
I20260808 13:10:48.162712 6113 tablet_copy_service.cc:161] P d82adb8c9a974543837b6ff639eae37c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c86d96cbfd324ad1ac8d9c4acfae8471 at {username='oryx', principal='oryx/127.31.59.189@KRBTEST.COM'} at 127.0.0.1:54680: session id = c86d96cbfd324ad1ac8d9c4acfae8471-00000000000000000000000000000000
I20260808 13:10:48.165957 6113 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 13:10:48.167887 6591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 13:10:48.172627 6591 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: tablet copy: Starting download of 0 data blocks...
I20260808 13:10:48.172932 6591 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: tablet copy: Starting download of 1 WAL segments...
I20260808 13:10:48.175416 6591 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 13:10:48.179333 6113 tablet_copy_service.cc:342] P d82adb8c9a974543837b6ff639eae37c: Request end of tablet copy session c86d96cbfd324ad1ac8d9c4acfae8471-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.59.189@KRBTEST.COM'} at 127.0.0.1:54680
I20260808 13:10:48.179531 6113 tablet_copy_service.cc:433] P d82adb8c9a974543837b6ff639eae37c: ending tablet copy session c86d96cbfd324ad1ac8d9c4acfae8471-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c86d96cbfd324ad1ac8d9c4acfae8471
I20260808 13:10:48.189600 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:48.192087 6710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:48.198282 6712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:48.203053 6714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:48.203332 6591 server_base.cc:1034] running on GCE node
I20260808 13:10:48.203779 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:48.204552 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:48.205817 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786194648205753 us; error 39 us; skew 500 ppm
I20260808 13:10:48.206130 6591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194648, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:48.207969 6591 init.cc:377] Logged in from keytab as oryx/127.31.59.189@KRBTEST.COM (short username oryx)
I20260808 13:10:48.210345 6591 webserver.cc:460] Webserver started at http://127.31.59.189:43223/ using document root <none> and password file <none>
I20260808 13:10:48.210647 6591 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:48.210731 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:48.213152 6591 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 13:10:48.214821 6721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:48.215200 6591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 13:10:48.215405 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "eb4b655efadb448a782633c0f821435c"
server_key_iv: "ef8a8c88ed4b2c4a967913d3c771170d"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.215581 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:48.235471 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:48.238749 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:48.262436 6591 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.189:33935
I20260808 13:10:48.262457 6774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.189:33935 every 8 connection(s)
I20260808 13:10:48.263353 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
I20260808 13:10:48.269259 6775 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:10:48.273090 6775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Bootstrap starting.
I20260808 13:10:48.291949 6775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:10:48.292474 6775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: Bootstrap complete.
I20260808 13:10:48.293609 6775 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } }
I20260808 13:10:48.293879 6775 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c86d96cbfd324ad1ac8d9c4acfae8471, State: Initialized, Role: LEARNER
I20260808 13:10:48.294145 6775 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } }
I20260808 13:10:48.294965 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } }
I20260808 13:10:48.295253 6775 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:48.295337 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:10:48.304301 6790 catalog_manager.cc:1259] Loaded cluster ID: 7a6fb6690c1d4b8abba1ec2d23a54360
I20260808 13:10:48.304507 6790 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: loading cluster ID for follower catalog manager: success
I20260808 13:10:48.308065 6790 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: acquiring CA information for follower catalog manager: success
I20260808 13:10:48.310317 6790 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:10:48.442518 6585 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
W20260808 13:10:48.498246 6701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:48.498546 6701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:48.498653 6701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:48.506381 6701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:48.506489 6701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:48.506542 6701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:48.506604 6701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:48.506649 6701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:48.517001 6701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.188.keytab
--principal=oryx/127.31.59.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:35963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:48.518484 6701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:48.519871 6701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:48.531418 6795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:48.531352 6796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:48.531599 6799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:48.532517 6701 server_base.cc:1034] running on GCE node
I20260808 13:10:48.533356 6701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:48.534912 6701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:48.536110 6701 hybrid_clock.cc:648] HybridClock initialized: now 1786194648536060 us; error 39 us; skew 500 ppm
I20260808 13:10:48.536432 6701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194648, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:48.539444 6701 init.cc:377] Logged in from keytab as oryx/127.31.59.188@KRBTEST.COM (short username oryx)
I20260808 13:10:48.540943 6701 webserver.cc:460] Webserver started at http://127.31.59.188:43053/ using document root <none> and password file <none>
I20260808 13:10:48.541673 6701 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:48.541795 6701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:48.542083 6701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:48.545013 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/instance:
uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
format_stamp: "Formatted at 2026-08-08 13:10:48 on dist-test-slave-49h3"
server_key: "a2ee4e4dfe96fbd76ed62af704118eee"
server_key_iv: "678db9df23858094f360b9afb6446162"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.545833 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal/instance:
uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
format_stamp: "Formatted at 2026-08-08 13:10:48 on dist-test-slave-49h3"
server_key: "a2ee4e4dfe96fbd76ed62af704118eee"
server_key_iv: "678db9df23858094f360b9afb6446162"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.551235 6701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:10:48.558027 6807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:48.559580 6701 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.007s sys 0.000s
I20260808 13:10:48.559795 6701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
format_stamp: "Formatted at 2026-08-08 13:10:48 on dist-test-slave-49h3"
server_key: "a2ee4e4dfe96fbd76ed62af704118eee"
server_key_iv: "678db9df23858094f360b9afb6446162"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.560092 6701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:48.581769 6701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:48.587502 6701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:48.587899 6701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:48.627449 6701 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.188:35963
I20260808 13:10:48.627530 6861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.188:35963 every 8 connection(s)
I20260808 13:10:48.629212 6701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
I20260808 13:10:48.633394 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6701
I20260808 13:10:48.634989 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:48.647583 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194648, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.188@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:48.667475 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.635880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.31.59.188:35963)
0808 13:10:48.637589 (+ 1709us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.637600 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.637683 (+ 83us) server_negotiation.cc:407] Connection header received
0808 13:10:48.641010 (+ 3327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.641034 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.641553 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.642088 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.645049 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.646739 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.649292 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.650128 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.650263 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.657059 (+ 6796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.657069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.657081 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.657134 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.661896 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.662698 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.662704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.662711 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.662912 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.663465 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.663469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.663471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.664160 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.664599 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.665027 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.665510 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388,"thread_start_us":989,"threads_started":1}
I20260808 13:10:48.672897 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.651372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.31.59.190:34673)
0808 13:10:48.651642 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.651649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.653255 (+ 1606us) server_negotiation.cc:407] Connection header received
0808 13:10:48.653481 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.653488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.653579 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.653689 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.655953 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.656847 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.659083 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.659703 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.659829 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.664266 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.664273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.664278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.664322 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.669080 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.670241 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.670248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.670252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.670463 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.670912 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.670916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.670918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.671459 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.671619 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.672305 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.672487 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 13:10:48.673204 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.651360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54686)
0808 13:10:48.652959 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.653010 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.653318 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.653888 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.653898 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.654978 (+ 1080us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.655784 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.655800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.657062 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.657066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.658926 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.658939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.659938 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.659946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.660133 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.660863 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.660880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.664026 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.669249 (+ 5223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.669268 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.669275 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.670110 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.670583 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.670587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.670590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.670790 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.671594 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.671601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.671901 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.672156 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.672475 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"thread_start_us":1114,"threads_started":1}
I20260808 13:10:48.675043 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194648, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.188@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
I20260808 13:10:48.696882 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.675782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55050)
0808 13:10:48.676026 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.676041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.676128 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.677939 (+ 1811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.677945 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.678669 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.678970 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.678982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.681133 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.681140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.683447 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.683464 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.684465 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.684471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.684786 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.685637 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.685655 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.688670 (+ 3015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.694056 (+ 5386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.694061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.694066 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.694539 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.694957 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.694962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.694966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.695161 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.695797 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.695802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.696003 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.696158 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.696493 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260808 13:10:48.697222 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.676261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.31.59.189:33935)
0808 13:10:48.677476 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.677483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.677506 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:10:48.677564 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.677568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.677655 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.677875 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.679146 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.680989 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.683613 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.684287 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.684416 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.688885 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.688890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.688894 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.688927 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.693933 (+ 5006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.694653 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.694659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.694661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.694845 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.695270 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.695276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.695279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.695654 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.695835 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.696366 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.696603 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":741,"threads_started":1}
I20260808 13:10:48.698657 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:10:48.701092 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.689665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55064)
0808 13:10:48.692105 (+ 2440us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.692127 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.692397 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.692897 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.692903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.692949 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:48.693227 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.693242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.695556 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.695562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.698203 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.698222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.700170 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.700177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.700377 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.700446 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.700541 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.700690 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2303,"thread_start_us":2130,"threads_started":1}
I20260808 13:10:48.701138 6870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.689999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.31.59.189:33935)
0808 13:10:48.691720 (+ 1721us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.691725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.692324 (+ 599us) server_negotiation.cc:407] Connection header received
0808 13:10:48.692522 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.692527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.692601 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.692848 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:48.693367 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.695380 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.698358 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.699974 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.700102 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.700447 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.700560 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.700734 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1434,"thread_start_us":1242,"threads_started":1}
I20260808 13:10:48.702836 6749 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 13:10:48.703545 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82adb8c9a974543837b6ff639eae37c. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } }
I20260808 13:10:48.703948 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: LEARNER
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194648, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.188@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
I20260808 13:10:48.715294 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.699400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.0.0.1:39456)
0808 13:10:48.699575 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.699596 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.699707 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.700207 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.700211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.700806 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.701084 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.701093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.702442 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.702445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.703953 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.703968 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.704928 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.704934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.705553 (+ 619us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.706202 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.706219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.708803 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.712316 (+ 3513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.712321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.712326 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.712791 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.713202 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.713206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.713209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.713540 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.714240 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.714263 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.714491 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.714639 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.714853 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:10:48.715377 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.699822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.31.59.188:35963)
0808 13:10:48.699978 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.699984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.700001 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:10:48.700037 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.700040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.700091 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.700175 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.701390 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.702326 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.704116 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.704723 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.704862 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.708977 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.708982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.708985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.709015 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.712164 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.712896 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.712901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.712903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.713052 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.713684 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.713688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.713690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.714117 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.714236 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.714740 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.715052 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":51}
I20260808 13:10:48.722114 6862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Bootstrap starting.
I20260808 13:10:48.727409 6862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:48.729120 6862 log.cc:824] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:48.733119 6862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: No bootstrap required, opened a new log
I20260808 13:10:48.744539 6862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } }
I20260808 13:10:48.744988 6862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:48.745235 6862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3bfccf7f11421694a2cbd57f1ddae6, State: Initialized, Role: FOLLOWER
I20260808 13:10:48.745913 6862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } }
I20260808 13:10:48.747666 6872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } } }
I20260808 13:10:48.748193 6872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:48.748822 6862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:48.752310 6872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:48.752794 6872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } }
I20260808 13:10:48.754980 6872 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673), c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189:33935)
I20260808 13:10:48.755741 6749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" is_pre_election: true
I20260808 13:10:48.755784 6102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d82adb8c9a974543837b6ff639eae37c" is_pre_election: true
I20260808 13:10:48.757309 6808 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f3bfccf7f11421694a2cbd57f1ddae6; no voters: c86d96cbfd324ad1ac8d9c4acfae8471, d82adb8c9a974543837b6ff639eae37c
I20260808 13:10:48.757994 6872 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:10:48.768343 6885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:10:48.768539 6885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:10:48.771596 6701 master_runner.cc:428] Detected that this master 1f3bfccf7f11421694a2cbd57f1ddae6 is joining an existing cluster
I20260808 13:10:48.771864 6701 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.59.188:35963
I20260808 13:10:48.775187 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6156
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.129@KRBTEST.COM
I20260808 13:10:48.820340 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.777412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.31.59.129:44869)
0808 13:10:48.790426 (+ 13014us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.790436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.790477 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:10:48.790559 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.790577 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.790923 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.791223 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.793392 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.795041 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.796825 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.797671 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.797797 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.807150 (+ 9353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.807172 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.807193 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.807245 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.814192 (+ 6947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.815819 (+ 1627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.815829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.815833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.816147 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.816780 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.816786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.816790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.817389 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.817826 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.819419 (+ 1593us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.819748 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12756,"thread_start_us":12565,"threads_started":1}
I20260808 13:10:48.822504 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.797550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.31.59.190:34673)
0808 13:10:48.797789 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.797795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.799638 (+ 1843us) server_negotiation.cc:407] Connection header received
0808 13:10:48.799949 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.799956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.800046 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.800120 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.802592 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.804192 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.807031 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.807883 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.807990 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.812334 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.812340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.812345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.812434 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.817633 (+ 5199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.818482 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.818488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.818491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.818727 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.819831 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.819837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.819840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.820199 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.820371 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.821794 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.822055 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:10:48.823123 6290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:48.824551 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.797696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54688)
0808 13:10:48.799405 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.799427 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.799786 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.800388 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.800392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.801753 (+ 1361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.802404 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.802420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.804427 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.804432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.806861 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.806878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.808346 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.808353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.808602 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.809604 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.809626 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.812096 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.817808 (+ 5712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.817813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.817818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.818335 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.818869 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.818874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.818878 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.819659 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.820446 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.820450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.820771 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.820937 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.821400 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"thread_start_us":1264,"threads_started":1}
W20260808 13:10:48.841975 6813 tablet.cc:2369] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:48.860268 6870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.800359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.31.59.189:33935)
0808 13:10:48.800549 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.800555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.810834 (+ 10279us) server_negotiation.cc:407] Connection header received
0808 13:10:48.811024 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.811029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.811117 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.811219 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.827909 (+ 16690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.829275 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.834398 (+ 5123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.835150 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.835298 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.850685 (+ 15387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.850692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.850695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.850742 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.856081 (+ 5339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.857153 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.857159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.857162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.857343 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.857858 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.857863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.857866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.858311 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.858477 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.859311 (+ 834us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.859673 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:10:48.863771 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.811415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.31.59.188:35963)
0808 13:10:48.811719 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.811726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.823567 (+ 11841us) server_negotiation.cc:407] Connection header received
0808 13:10:48.823790 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.823795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.823872 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.823985 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.829658 (+ 5673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.831488 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.843971 (+ 12483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.844977 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.845089 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.852201 (+ 7112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.852207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.852212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.852245 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.857989 (+ 5744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.859097 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.859101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.859104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.859419 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.859911 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.859916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.859919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.860776 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.860981 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.861958 (+ 977us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.863249 (+ 1291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1097,"server-negotiator.queue_time_us":64}
I20260808 13:10:48.864775 6894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.799610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55072)
0808 13:10:48.810541 (+ 10931us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.810580 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.810888 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.813701 (+ 2813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.813706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.820178 (+ 6472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.827702 (+ 7524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.827719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.829515 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.829525 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.834214 (+ 4689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.834229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.835805 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.835815 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.836117 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.837691 (+ 1574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.837719 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.849613 (+ 11894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.856321 (+ 6708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.856327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.856330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.857002 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.857527 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.857532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.857535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.857708 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.858617 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.858622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.858968 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.859165 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.862225 (+ 3060us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10639,"thread_start_us":11377,"threads_started":1}
I20260808 13:10:48.865146 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.811653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.0.0.1:39472)
0808 13:10:48.823231 (+ 11578us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.823309 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.823672 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.824187 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.824192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.826505 (+ 2313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.828673 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.828689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.836985 (+ 8296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.836991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.843761 (+ 6770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.843782 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.845703 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.845714 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.845963 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.847174 (+ 1211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.847530 (+ 356us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.852004 (+ 4474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.858118 (+ 6114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.858122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.858126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.858979 (+ 853us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.859538 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.859543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.859547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.859804 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.861036 (+ 1232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.861041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.861248 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.861490 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.863070 (+ 1580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11405,"mutex_wait_us":903,"thread_start_us":11037,"threads_started":1}
Aug 08 13:10:48 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.190@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
I20260808 13:10:48.882175 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.835896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.0.0.1:39474)
0808 13:10:48.836275 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.836295 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.836407 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.849011 (+ 12604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.849018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.849766 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.850166 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.850181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.860917 (+ 10736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.860924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.862687 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:48.862706 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.863926 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.863933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:48.864160 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.864992 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:48.865017 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:48.869533 (+ 4516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:48.878476 (+ 8943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.878483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.878487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.879312 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.879904 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:48.879909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:48.879913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:48.880139 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:48.880913 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:48.880919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:48.881191 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 13:10:48.881341 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 13:10:48.881704 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250}
I20260808 13:10:48.882859 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.836660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.31.59.188:35963)
0808 13:10:48.848343 (+ 11683us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.848349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.848414 (+ 65us) server_negotiation.cc:407] Connection header received
0808 13:10:48.848529 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.848535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.848646 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.848918 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:48.850417 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.860757 (+ 10340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.862822 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:48.863738 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:48.864108 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:48.869863 (+ 5755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:48.869872 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:48.869878 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:48.869922 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:48.878248 (+ 8326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.879516 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.879522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.879526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.879728 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:48.880309 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:48.880313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:48.880317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:48.880760 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:48.881603 (+ 843us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:48.881821 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 13:10:48.882051 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11483,"thread_start_us":10775,"threads_started":1}
I20260808 13:10:48.884532 6092 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.59.188:35963: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
member_type: NON_VOTER
last_known_addr {
host: "127.31.59.188"
port: 35963
}
attrs {
promote: true
}
}
dest_uuid: "d82adb8c9a974543837b6ff639eae37c"
cas_config_opid_index: 5
I20260808 13:10:48.885974 6092 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } }
I20260808 13:10:48.888003 6868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.893244 6868 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.59.188:35963
I20260808 13:10:48.894174 6868 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.894556 6868 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:48.894582 6749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 13:10:48.895071 6701 master.cc:537] Master@127.31.59.188:35963 shutting down...
I20260808 13:10:48.896884 6898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 13:10:48.901697 6749 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.908535 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: NON_VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: true } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.909054 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:10:48.897149 6834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:48.914049 6898 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: attempting to promote NON_VOTER c86d96cbfd324ad1ac8d9c4acfae8471 to VOTER
I20260808 13:10:48.915246 6898 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } }
I20260808 13:10:48.919206 6749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 LEARNER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 13:10:48.919932 6868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 13:10:48.923688 6834 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 13:10:48.923975 6749 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.928149 6901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.932921 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.933323 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:48.934195 6868 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.934568 6868 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:48.934113 6901 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:48.934971 6901 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
W20260808 13:10:48.964619 6075 consensus_peers.cc:487] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c -> Peer 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188:35963): Couldn't send request to peer 1f3bfccf7f11421694a2cbd57f1ddae6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:10:48.970027 6701 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:48.970866 6701 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:48.972049 6701 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: stopping tablet replica
I20260808 13:10:48.972740 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6156
I20260808 13:10:48.979322 6701 master.cc:552] Master@127.31.59.188:35963 shutdown complete.
I20260808 13:10:48.979414 6701 master_runner.cc:305] Clearing existing system tablet
I20260808 13:10:48.979743 6701 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:48.979842 6701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:48.982136 6701 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260808 13:10:48.983590 6905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:48.983901 6701 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:48.984153 6701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 13:10:48.984251 6701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
format_stamp: "Formatted at 2026-08-08 13:10:48 on dist-test-slave-49h3"
server_key: "a2ee4e4dfe96fbd76ed62af704118eee"
server_key_iv: "678db9df23858094f360b9afb6446162"
server_key_version: "encryptionkey@0"
I20260808 13:10:48.987666 6701 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:48.993680 6701 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 13:10:48.993901 6701 log.cc:1196] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 13:10:48.994561 6701 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Deleting consensus metadata
I20260808 13:10:48.994843 6701 master_runner.cc:315] Copying system tablet from 127.31.59.190:34673
I20260808 13:10:48.997262 6701 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: tablet copy: Beginning tablet copy session from remote peer at address 127.31.59.190:34673
I20260808 13:10:49.001737 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:44869
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:34673
--webserver_port=33219
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:10:49.010593 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.998204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.31.59.190:34673)
0808 13:10:48.998466 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:48.998473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:48.998925 (+ 452us) server_negotiation.cc:407] Connection header received
0808 13:10:48.999118 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:48.999124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:48.999195 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:48.999276 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:49.000147 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.001059 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.002656 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.003081 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.003195 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.005036 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:49.005043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:49.005046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:49.005082 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:49.007990 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.008633 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.008639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.008643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.008790 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.009156 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.009161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.009164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.009441 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:49.009616 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.009999 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.010232 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 13:10:49.010651 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:48.998045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:54704)
0808 13:10:48.998774 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:48.998798 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:48.998981 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:48.999329 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:48.999334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:48.999767 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:48.999992 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.000001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.001202 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.001207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.002513 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.002526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.003238 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.003244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.003415 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.004017 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:49.004039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:49.004863 (+ 824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:49.008136 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.008143 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.008147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.008521 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.008899 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.008905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.008908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.009054 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.009590 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:49.009595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:49.009754 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 13:10:49.009875 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 13:10:49.010227 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":506,"threads_started":1}
I20260808 13:10:49.011694 6113 tablet_copy_service.cc:140] P d82adb8c9a974543837b6ff639eae37c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1f3bfccf7f11421694a2cbd57f1ddae6 ({username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:54704)
I20260808 13:10:49.011905 6113 tablet_copy_service.cc:161] P d82adb8c9a974543837b6ff639eae37c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1f3bfccf7f11421694a2cbd57f1ddae6 at {username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:54704: session id = 1f3bfccf7f11421694a2cbd57f1ddae6-00000000000000000000000000000000
I20260808 13:10:49.015583 6113 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 13:10:49.017570 6701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 13:10:49.024210 6701 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: tablet copy: Starting download of 0 data blocks...
I20260808 13:10:49.024530 6701 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: tablet copy: Starting download of 1 WAL segments...
I20260808 13:10:49.027418 6701 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 13:10:49.032485 6113 tablet_copy_service.cc:342] P d82adb8c9a974543837b6ff639eae37c: Request end of tablet copy session 1f3bfccf7f11421694a2cbd57f1ddae6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:54704
I20260808 13:10:49.032649 6113 tablet_copy_service.cc:433] P d82adb8c9a974543837b6ff639eae37c: ending tablet copy session 1f3bfccf7f11421694a2cbd57f1ddae6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1f3bfccf7f11421694a2cbd57f1ddae6
I20260808 13:10:49.044380 6701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:49.047073 6914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:49.050822 6915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:49.058135 6701 server_base.cc:1034] running on GCE node
W20260808 13:10:49.058458 6918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:49.058946 6701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:49.059710 6701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:49.060961 6701 hybrid_clock.cc:648] HybridClock initialized: now 1786194649060903 us; error 42 us; skew 500 ppm
I20260808 13:10:49.061204 6701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194649, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:49.063493 6701 init.cc:377] Logged in from keytab as oryx/127.31.59.188@KRBTEST.COM (short username oryx)
I20260808 13:10:49.069597 6701 webserver.cc:460] Webserver started at http://127.31.59.188:43533/ using document root <none> and password file <none>
I20260808 13:10:49.069906 6701 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:49.070003 6701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:49.072402 6701 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 13:10:49.074160 6925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:49.074568 6701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 13:10:49.074681 6701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "1f3bfccf7f11421694a2cbd57f1ddae6"
format_stamp: "Formatted at 2026-08-08 13:10:48 on dist-test-slave-49h3"
server_key: "a2ee4e4dfe96fbd76ed62af704118eee"
server_key_iv: "678db9df23858094f360b9afb6446162"
server_key_version: "encryptionkey@0"
I20260808 13:10:49.074862 6701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:49.094244 6701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:49.098035 6701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:49.137565 6701 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.188:35963
I20260808 13:10:49.137744 6978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.188:35963 every 8 connection(s)
I20260808 13:10:49.138494 6701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
I20260808 13:10:49.144202 6979 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:10:49.148674 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Bootstrap starting.
I20260808 13:10:49.163956 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:10:49.164439 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Bootstrap complete.
I20260808 13:10:49.165385 6979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } }
I20260808 13:10:49.165671 6979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1f3bfccf7f11421694a2cbd57f1ddae6, State: Initialized, Role: LEARNER
I20260808 13:10:49.165926 6979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } }
I20260808 13:10:49.166842 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:49.167111 6979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:49.167326 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:10:49.175462 6995 catalog_manager.cc:1259] Loaded cluster ID: 7a6fb6690c1d4b8abba1ec2d23a54360
I20260808 13:10:49.175566 6995 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: loading cluster ID for follower catalog manager: success
I20260808 13:10:49.178297 6995 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: acquiring CA information for follower catalog manager: success
I20260808 13:10:49.180172 6995 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:10:49.315552 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.300713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.0.0.1:39488)
0808 13:10:49.301011 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:49.301032 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:49.301205 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:49.306544 (+ 5339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:49.306551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:49.306585 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:49.306852 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.306871 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.309369 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.309375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.311602 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.311622 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.314007 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.314015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.314363 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.314406 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 13:10:49.314544 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 13:10:49.314877 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 13:10:49.316859 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.300904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39488 (local address 127.31.59.188:35963)
0808 13:10:49.306078 (+ 5174us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:49.306085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:49.306110 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:10:49.306195 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:49.306201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:49.306306 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:49.306514 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:49.307042 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.309182 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.311815 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.313742 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.313876 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.314237 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.315894 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.316059 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5000,"thread_start_us":4738,"threads_started":1}
I20260808 13:10:49.318392 6952 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 13:10:49.319473 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82adb8c9a974543837b6ff639eae37c. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:49.319921 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: LEARNER
W20260808 13:10:49.460777 6913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:49.461138 6913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:49.461318 6913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:49.471935 6913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:49.472092 6913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:49.472317 6913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:49.485375 6913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:44869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=33219
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:49.486879 6913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:49.488582 6913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:49.506948 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:49.506958 7005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:49.507072 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:49.508550 6913 server_base.cc:1034] running on GCE node
I20260808 13:10:49.509374 6913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:49.510757 6913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:49.512045 6913 hybrid_clock.cc:648] HybridClock initialized: now 1786194649511929 us; error 45 us; skew 500 ppm
I20260808 13:10:49.512538 6913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194649, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:49.517370 6913 init.cc:377] Logged in from keytab as oryx/127.31.59.129@KRBTEST.COM (short username oryx)
I20260808 13:10:49.519377 6913 webserver.cc:460] Webserver started at http://127.31.59.129:33219/ using document root <none> and password file <none>
I20260808 13:10:49.520179 6913 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:49.520426 6913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:49.527279 6913 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 13:10:49.530632 7012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:49.532078 6913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:49.532271 6913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "c6f04941a99e4fe38adba3e29fbdbf18"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "5caafd6d34558593e2ef8060a9611576"
server_key_iv: "d8f06dc3a4f6df533c73bda0b16c9736"
server_key_version: "encryptionkey@0"
I20260808 13:10:49.533682 6913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:49.566103 6913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:49.572196 6913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:49.572650 6913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:49.574115 6913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:49.576189 6913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:49.576400 6913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:49.576575 6913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:49.576702 6913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:49.654657 6913 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44869
I20260808 13:10:49.654902 7132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44869 every 8 connection(s)
I20260808 13:10:49.656481 6913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:49.666896 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 6913
I20260808 13:10:49.667268 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6303
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194649, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194649, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194649, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
Aug 08 13:10:49 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.130@KRBTEST.COM
I20260808 13:10:49.692288 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.659817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43551 (local address 127.31.59.189:33935)
0808 13:10:49.660691 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:49.660696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:49.661758 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 13:10:49.666115 (+ 4357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:49.666121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:49.666213 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:49.666301 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:49.669917 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.671306 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.674724 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.675455 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.675601 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.682512 (+ 6911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:49.682519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:49.682523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:49.682589 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:49.687693 (+ 5104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.688898 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.688904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.688907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.689081 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.690205 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.690210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.690213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.690701 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:49.690846 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.691548 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.691786 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":633,"threads_started":1}
I20260808 13:10:49.693933 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.660422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.31.59.129:43551)
0808 13:10:49.661579 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:49.661630 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:49.665768 (+ 4138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:49.666776 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:49.666780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:49.667948 (+ 1168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:49.669692 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.669721 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.671628 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.671635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.674569 (+ 2934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.674589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.675838 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.675848 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.676267 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.677926 (+ 1659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:49.677955 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:49.682161 (+ 4206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:49.687818 (+ 5657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.687823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.687826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.688743 (+ 917us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.689850 (+ 1107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.689855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.689858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.690076 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.690908 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:49.690912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:49.691104 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 13:10:49.692568 (+ 1464us) client_negotiation.cc:240] Negotiation successful
0808 13:10:49.693007 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":97,"spinlock_wait_cycles":3840,"thread_start_us":491,"threads_started":1}
I20260808 13:10:49.694283 7141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.662103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54659 (local address 127.31.59.190:34673)
0808 13:10:49.664184 (+ 2081us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:49.664191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:49.664215 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:10:49.666074 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:49.666080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:49.666193 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:49.666302 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:49.670390 (+ 4088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.671758 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.675927 (+ 4169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.676764 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.676881 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.682809 (+ 5928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:49.682815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:49.682818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:49.682854 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:49.687416 (+ 4562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.688629 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.688635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.688638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.688849 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.690541 (+ 1692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.690547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.690550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.690972 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:49.691128 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.693640 (+ 2512us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.693879 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1878,"thread_start_us":1759,"threads_started":1}
I20260808 13:10:49.694448 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.661907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:54659)
0808 13:10:49.663312 (+ 1405us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:49.663332 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:49.665773 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:49.667765 (+ 1992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:49.667769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:49.668808 (+ 1039us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:49.669893 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.669906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.671910 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.671915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.674604 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.674627 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.677058 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.677067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.677320 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.678237 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:49.678265 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:49.682521 (+ 4256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:49.687552 (+ 5031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.687562 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.687582 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.688442 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.689226 (+ 784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.689231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.689235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.689717 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.691081 (+ 1364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:49.691085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:49.691264 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 13:10:49.691456 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 13:10:49.692041 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"mutex_wait_us":103,"thread_start_us":767,"threads_started":1}
I20260808 13:10:49.694748 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.663419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:40063 (local address 127.31.59.188:35963)
0808 13:10:49.663642 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:49.663649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:49.664898 (+ 1249us) server_negotiation.cc:407] Connection header received
0808 13:10:49.666105 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:49.666112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:49.666198 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:49.666274 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:49.670499 (+ 4225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.671898 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.674817 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.675611 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.675724 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.682767 (+ 7043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:49.682774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:49.682778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:49.682821 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:49.687641 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.689056 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.689062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.689064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.689315 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.689852 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.689858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.689861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.690262 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:49.690466 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.693996 (+ 3530us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.694277 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 13:10:49.695765 7142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.663779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.31.59.129:40063)
0808 13:10:49.664782 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:49.664800 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:49.665773 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:49.666759 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:49.666771 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:49.669166 (+ 2395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:49.670326 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.670341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.672097 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.672101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.674615 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:49.674625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.676031 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.676039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:49.676260 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.677223 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:49.677248 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:49.682505 (+ 5257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:49.687805 (+ 5300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.687811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.687815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.688925 (+ 1110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.689490 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:49.689495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:49.689498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:49.689735 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:49.690475 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:49.690485 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:49.690981 (+ 496us) client_negotiation.cc:769] Sending connection context
0808 13:10:49.694912 (+ 3931us) client_negotiation.cc:240] Negotiation successful
0808 13:10:49.695159 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":149,"spinlock_wait_cycles":24320,"thread_start_us":597,"threads_started":1}
I20260808 13:10:49.696544 7136 heartbeater.cc:347] Connected to a master server at 127.31.59.189:33935
I20260808 13:10:49.696931 7136 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:49.698240 7133 heartbeater.cc:347] Connected to a master server at 127.31.59.188:35963
I20260808 13:10:49.698593 7133 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:49.699939 7136 heartbeater.cc:511] Master 127.31.59.189:33935 requested a full tablet report, sending...
I20260808 13:10:49.700084 7133 heartbeater.cc:511] Master 127.31.59.188:35963 requested a full tablet report, sending...
I20260808 13:10:49.702776 6942 ts_manager.cc:194] Registered new tserver with Master: c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:49.702704 6739 ts_manager.cc:194] Registered new tserver with Master: c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:49.705931 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:49.668902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.31.59.130:43315)
0808 13:10:49.670492 (+ 1590us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:49.670500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:49.670528 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:10:49.670597 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:49.670606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:49.670845 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:49.670998 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:49.674876 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.677849 (+ 2973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.680379 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:49.681385 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:49.681509 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:49.691562 (+ 10053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:49.691576 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:49.691589 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:49.691622 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:49.701906 (+ 10284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.703107 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.703118 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.703130 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.703343 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:49.703822 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:49.703828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:49.703832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:49.704324 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:49.704685 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:49.704944 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 13:10:49.705233 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388,"thread_start_us":1183,"threads_started":1}
I20260808 13:10:49.706709 7134 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:49.706904 7134 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:49.707569 6435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:49.708047 7134 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:49.709815 6092 ts_manager.cc:194] Re-registered known tserver with Master: c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:49.711207 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:54659
I20260808 13:10:49.714246 6901 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: NON_VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: true } } }
I20260808 13:10:49.714783 6901 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:49.723531 7147 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: attempting to promote NON_VOTER 1f3bfccf7f11421694a2cbd57f1ddae6 to VOTER
I20260808 13:10:49.724371 7147 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } }
I20260808 13:10:49.727294 6952 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 LEARNER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 13:10:49.727363 6749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Refusing update from remote peer d82adb8c9a974543837b6ff639eae37c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 13:10:49.727985 6901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 13:10:49.728516 7148 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 13:10:49.731693 6901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.732453 6749 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.738070 6901 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.739140 6901 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:49.738893 7149 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.740008 7149 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:49.744714 7148 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.745334 7148 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:49.745818 6952 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.750959 7152 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.751473 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d82adb8c9a974543837b6ff639eae37c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:49.751775 7152 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:49.751894 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:49.824530 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6303
I20260808 13:10:49.841995 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:43315
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:34673
--webserver_port=34675
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:50.258010 7153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:50.258288 7153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:50.258417 7153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:50.265623 7153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:50.265736 7153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:50.265884 7153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:50.275785 7153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:43315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=34675
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:50.277083 7153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:50.278493 7153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:50.291419 7158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:50.291378 7161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:50.291347 7159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:50.292306 7153 server_base.cc:1034] running on GCE node
I20260808 13:10:50.292855 7153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:50.293802 7153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:50.295001 7153 hybrid_clock.cc:648] HybridClock initialized: now 1786194650294955 us; error 38 us; skew 500 ppm
I20260808 13:10:50.295305 7153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:50.298460 7153 init.cc:377] Logged in from keytab as oryx/127.31.59.130@KRBTEST.COM (short username oryx)
I20260808 13:10:50.299909 7153 webserver.cc:460] Webserver started at http://127.31.59.130:34675/ using document root <none> and password file <none>
I20260808 13:10:50.300652 7153 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:50.300789 7153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:50.306135 7153 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:50.308729 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.310014 7153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:10:50.310165 7153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "591352facb7143999b896ae541d3f2c5"
format_stamp: "Formatted at 2026-08-08 13:10:46 on dist-test-slave-49h3"
server_key: "9dfa9637d823b9d2eae9a8a64cf41580"
server_key_iv: "cf185eb68c5eba46938d253b0a9dbc8f"
server_key_version: "encryptionkey@0"
I20260808 13:10:50.311100 7153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:50.338676 7153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:50.343042 7153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:50.343338 7153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:50.344381 7153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:50.345818 7153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:50.345920 7153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.346025 7153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:50.346108 7153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.377003 7153 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:43315
I20260808 13:10:50.377072 7287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:43315 every 8 connection(s)
I20260808 13:10:50.378544 7153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:50.379485 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 7153
I20260808 13:10:50.379814 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6445
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.131@KRBTEST.COM
I20260808 13:10:50.406905 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.380912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.31.59.131:45791)
0808 13:10:50.385081 (+ 4169us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.385088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.385113 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:10:50.385191 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.385197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.385377 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.385576 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.387877 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.389716 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.391795 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.392788 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.394145 (+ 1357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.399236 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:50.399252 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:50.399267 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:50.399314 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:50.403128 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.403980 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.403989 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.403992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.404172 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.404633 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.404638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.404641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.405060 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:50.405339 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:50.405983 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 13:10:50.406290 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4007,"thread_start_us":3852,"threads_started":1}
I20260808 13:10:50.408967 6579 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
I20260808 13:10:50.419353 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.381417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:60693 (local address 127.31.59.188:35963)
0808 13:10:50.393734 (+ 12317us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.393741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.395520 (+ 1779us) server_negotiation.cc:407] Connection header received
0808 13:10:50.398699 (+ 3179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.398707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.398808 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.398983 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.401323 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.402469 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.404929 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.405538 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.405661 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.411284 (+ 5623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:50.411292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:50.411298 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:50.411343 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:50.415178 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.416408 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.416414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.416418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.416597 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.417168 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.417174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.417178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.417572 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:50.417700 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:50.418662 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 13:10:50.418985 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12157,"thread_start_us":12885,"threads_started":1}
I20260808 13:10:50.422005 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.382138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.31.59.130:60693)
0808 13:10:50.395304 (+ 13166us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.395369 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.398192 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.399465 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.399478 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.400243 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.401105 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.401135 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.402681 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.402687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.404747 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.404779 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.405963 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.405971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.406302 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.406956 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:50.407266 (+ 310us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:50.411008 (+ 3742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:50.415355 (+ 4347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.415366 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.415385 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.416255 (+ 870us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.416742 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.416748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.416753 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.417031 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.417805 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:50.417813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:50.418236 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 13:10:50.419936 (+ 1700us) client_negotiation.cc:240] Negotiation successful
0808 13:10:50.420483 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12797,"thread_start_us":12462,"threads_started":1}
I20260808 13:10:50.428130 7288 heartbeater.cc:347] Connected to a master server at 127.31.59.188:35963
I20260808 13:10:50.428438 7288 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:50.429415 7288 heartbeater.cc:511] Master 127.31.59.188:35963 requested a full tablet report, sending...
I20260808 13:10:50.431373 6942 ts_manager.cc:194] Registered new tserver with Master: 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:50.440759 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.396022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:53979 (local address 127.31.59.189:33935)
0808 13:10:50.425933 (+ 29911us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.425939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.425957 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:10:50.426021 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.426026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.426127 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.426246 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.427759 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.428810 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.430654 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.431259 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.431360 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.434921 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:50.434929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:50.434935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:50.434977 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:50.438261 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.438864 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.438868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.438870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.439002 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.439410 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.439414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.439416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.439701 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:50.439831 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:50.440206 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 13:10:50.440425 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29769,"thread_start_us":29618,"threads_started":1}
I20260808 13:10:50.440769 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.395879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.31.59.130:53979)
0808 13:10:50.422764 (+ 26885us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.422785 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.422899 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.426289 (+ 3390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.426293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.427353 (+ 1060us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.427559 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.427573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.429033 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.429038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.430488 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.430501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.431608 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.431614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.431825 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.432488 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:50.432506 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:50.434708 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:50.438381 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.438386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.438391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.438738 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.439166 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.439171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.439176 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.439303 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.439829 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:50.439834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:50.439968 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 13:10:50.440081 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 13:10:50.440356 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26716,"thread_start_us":29997,"threads_started":1}
I20260808 13:10:50.441826 7290 heartbeater.cc:347] Connected to a master server at 127.31.59.189:33935
I20260808 13:10:50.441982 7290 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:50.442454 7290 heartbeater.cc:511] Master 127.31.59.189:33935 requested a full tablet report, sending...
I20260808 13:10:50.443359 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.426403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:46589)
0808 13:10:50.427238 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.427257 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.427376 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.427966 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.427971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.428794 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.429160 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.429174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.430695 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.430701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.432317 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.432333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.433620 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.433628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.433791 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.434888 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:50.434904 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:50.436822 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:50.440671 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.440677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.440681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.441162 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.441595 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:50.441600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:50.441604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:50.441760 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:50.442379 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:50.442384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:50.442560 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 13:10:50.442709 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 13:10:50.442986 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690}
I20260808 13:10:50.443543 6739 ts_manager.cc:194] Registered new tserver with Master: 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
I20260808 13:10:50.444135 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.426523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:46589 (local address 127.31.59.190:34673)
0808 13:10:50.427045 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.427050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.427477 (+ 427us) server_negotiation.cc:407] Connection header received
0808 13:10:50.427547 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.427552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.427656 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.427779 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.429351 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.430525 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.432492 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.433221 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.433339 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.436999 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:50.437005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:50.437010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:50.437062 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:50.440510 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.441299 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.441305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.441309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.441474 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:50.441861 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:50.441866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:50.441869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:50.442227 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:50.442390 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:50.442828 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 13:10:50.443789 (+ 961us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":278,"threads_started":1}
I20260808 13:10:50.444813 7289 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:50.444931 7289 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:50.445313 7289 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:50.446342 6092 ts_manager.cc:194] Re-registered known tserver with Master: 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
I20260808 13:10:50.447410 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:46589
I20260808 13:10:50.509230 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6445
I20260808 13:10:50.526683 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:45791
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:34673
--webserver_port=45167
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:10:50.715107 7134 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
W20260808 13:10:50.861274 7301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:50.861611 7301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:50.861768 7301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:50.869024 7301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:50.869208 7301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:50.869365 7301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:50.879650 7301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:45791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=45167
--tserver_master_addrs=127.31.59.190:34673,127.31.59.189:33935,127.31.59.188:35963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:50.881064 7301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:50.882464 7301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:50.895490 7307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:50.895519 7309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:50.895565 7306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:50.896840 7301 server_base.cc:1034] running on GCE node
I20260808 13:10:50.897574 7301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:50.898581 7301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:50.899772 7301 hybrid_clock.cc:648] HybridClock initialized: now 1786194650899713 us; error 52 us; skew 500 ppm
I20260808 13:10:50.900113 7301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:50.907480 7301 init.cc:377] Logged in from keytab as oryx/127.31.59.131@KRBTEST.COM (short username oryx)
I20260808 13:10:50.909011 7301 webserver.cc:460] Webserver started at http://127.31.59.131:45167/ using document root <none> and password file <none>
I20260808 13:10:50.909801 7301 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:50.909938 7301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:50.915530 7301 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:10:50.918354 7317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.919634 7301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:10:50.919792 7301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8c7dd9c99c574d0489816e0505962ff8"
format_stamp: "Formatted at 2026-08-08 13:10:47 on dist-test-slave-49h3"
server_key: "e192953a6b198d3a8d840e8b2619887b"
server_key_iv: "99a47f76a734739a1dbe51bcbb5af293"
server_key_version: "encryptionkey@0"
I20260808 13:10:50.920856 7301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:50.935783 7301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:50.940011 7301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:50.940282 7301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:50.941303 7301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:10:50.942714 7301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:50.942812 7301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.942924 7301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:50.942997 7301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:50.974390 7301 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:45791
I20260808 13:10:50.974452 7439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:45791 every 8 connection(s)
I20260808 13:10:50.975978 7301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:50.980850 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 7301
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.188@KRBTEST.COM
Aug 08 13:10:50 dist-test-slave-49h3 krb5kdc[6043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194650, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.189@KRBTEST.COM
I20260808 13:10:51.010043 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.978813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:48531 (local address 127.31.59.190:34673)
0808 13:10:50.979237 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.979241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.980239 (+ 998us) server_negotiation.cc:407] Connection header received
0808 13:10:50.987463 (+ 7224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.987470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.987578 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.987664 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.990898 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.992120 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.995815 (+ 3695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.996516 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.996617 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.002252 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.002260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.002265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.002311 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.005973 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.007003 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.007009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.007013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.007214 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.007791 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.007797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.007799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.008168 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.008309 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.009386 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.009639 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":216,"threads_started":1}
I20260808 13:10:51.012497 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.979265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:48531)
0808 13:10:50.979993 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.980040 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.987285 (+ 7245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.988244 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.988257 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.989670 (+ 1413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.990621 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.990649 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.992377 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.992383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.994738 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.994763 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.997047 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.997054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.997403 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:50.998304 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:50.998341 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:51.001920 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:51.006221 (+ 4301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.006231 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.006252 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.006857 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.007380 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.007386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.007389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.007646 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.008410 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:51.008418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:51.008909 (+ 491us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.010394 (+ 1485us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.010970 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":290,"threads_started":1}
I20260808 13:10:51.014176 7441 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34673
I20260808 13:10:51.014590 7441 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:51.015647 7441 heartbeater.cc:511] Master 127.31.59.190:34673 requested a full tablet report, sending...
I20260808 13:10:51.016136 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.980795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:46407 (local address 127.31.59.189:33935)
0808 13:10:50.992653 (+ 11858us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.992660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.992684 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:10:50.992761 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.992766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.992848 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.992971 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.994370 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.995238 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.997819 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.998481 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.998599 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.005373 (+ 6774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.005381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.005385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.005428 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.009948 (+ 4520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.013599 (+ 3651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.013605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.013608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.013769 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.014200 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.014205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.014208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.014518 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.014665 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.015544 (+ 879us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.015773 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11704,"thread_start_us":11597,"threads_started":1}
I20260808 13:10:51.016487 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.980488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.31.59.131:46407)
0808 13:10:50.981086 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.981104 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.986789 (+ 5685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.993022 (+ 6233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.993028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.993856 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.994212 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.994227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.995427 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.995431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.996990 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.997004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.999802 (+ 2798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.999809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.999985 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.000841 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:51.000896 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:51.005131 (+ 4235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:51.012830 (+ 7699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.012835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.012838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.013402 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.013904 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.013909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.013912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.014055 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.014782 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:51.014786 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:51.014947 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.015116 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.015528 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":544,"threads_started":1}
I20260808 13:10:51.017916 6092 ts_manager.cc:194] Re-registered known tserver with Master: 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791)
I20260808 13:10:51.019196 6092 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:48531
I20260808 13:10:51.019273 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.981601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:47831 (local address 127.31.59.188:35963)
0808 13:10:50.995273 (+ 13672us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:50.995280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:50.995300 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:50.995366 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:50.995371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:50.995457 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:50.995571 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:50.996720 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.997662 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.999694 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.000308 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.001039 (+ 731us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.007523 (+ 6484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.007542 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.007547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.007589 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.012659 (+ 5070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.016419 (+ 3760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.016425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.016428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.016715 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.017647 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.017651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.017654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.018164 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.018305 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.018709 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.018934 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13539,"thread_start_us":13419,"threads_started":1}
I20260808 13:10:51.019299 7448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:50.981490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.31.59.131:47831)
0808 13:10:50.982025 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:50.982043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:50.987214 (+ 5171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:50.995619 (+ 8405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:50.995623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:50.996213 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:50.996532 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.996544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:50.997789 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:50.997794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:50.999528 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:50.999543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.000517 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.000524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.000739 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.001275 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:51.001297 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:51.004399 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:51.014763 (+ 10364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.014768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.014771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.015323 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.016973 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:51.016979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:51.016982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:51.017212 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:51.018295 (+ 1083us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:51.018300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:51.018442 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.018575 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.018855 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":278,"threads_started":1}
I20260808 13:10:51.019604 7442 heartbeater.cc:347] Connected to a master server at 127.31.59.189:33935
I20260808 13:10:51.019752 7442 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:51.020118 7440 heartbeater.cc:347] Connected to a master server at 127.31.59.188:35963
I20260808 13:10:51.020238 7440 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:51.020265 7442 heartbeater.cc:511] Master 127.31.59.189:33935 requested a full tablet report, sending...
I20260808 13:10:51.020582 7440 heartbeater.cc:511] Master 127.31.59.188:35963 requested a full tablet report, sending...
I20260808 13:10:51.021674 6739 ts_manager.cc:194] Registered new tserver with Master: 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791)
I20260808 13:10:51.021699 6942 ts_manager.cc:194] Registered new tserver with Master: 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791)
I20260808 13:10:51.041383 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.003745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.31.59.190:34673)
0808 13:10:51.010320 (+ 6575us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.010326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.024852 (+ 14526us) server_negotiation.cc:407] Connection header received
0808 13:10:51.025111 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.025116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.025188 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.025278 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.028453 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.029571 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.031288 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.031676 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.031790 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.035520 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.035527 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.035531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.035573 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.038576 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.039215 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.039221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.039225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.039345 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.039780 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.039785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.039788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.040171 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.040433 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.040825 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.041048 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6468,"thread_start_us":23572,"threads_started":1}
I20260808 13:10:51.042850 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.026520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.31.59.188:35963)
0808 13:10:51.026726 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.026731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.027497 (+ 766us) server_negotiation.cc:407] Connection header received
0808 13:10:51.027759 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.027764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.027867 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.027969 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.029890 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.030923 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.032621 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.033030 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.033102 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.036129 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.036135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.036139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.036176 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.039568 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.040409 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.040414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.040417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.040563 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.040964 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.040969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.040972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.041346 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.041858 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.042382 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.042559 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:10:51.044148 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.025547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.31.59.189:33935)
0808 13:10:51.025703 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.025709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.026556 (+ 847us) server_negotiation.cc:407] Connection header received
0808 13:10:51.026795 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.026798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.026858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.026943 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.029584 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.030380 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.032014 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.032537 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.032660 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.036093 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.036099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.036103 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.036141 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.039384 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.040070 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.040076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.040080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.040210 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.040638 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.040644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.040647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.041246 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.041838 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.043593 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.043831 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 13:10:51.046262 6092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:51.053153 6092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:51.092556 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.075651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:45791 (local address 127.0.0.1:37068)
0808 13:10:51.078111 (+ 2460us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:51.078155 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:51.078294 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:51.080062 (+ 1768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:51.080068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:51.080097 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:51.081209 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.081225 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.084098 (+ 2873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.084105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.088793 (+ 4688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.088808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.091360 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.091366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.091620 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.091649 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.091737 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.091895 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"thread_start_us":12588,"threads_started":1}
I20260808 13:10:51.092624 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.077697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37068 (local address 127.31.59.131:45791)
0808 13:10:51.079329 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.079339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.079413 (+ 74us) server_negotiation.cc:407] Connection header received
0808 13:10:51.079535 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.079544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.079848 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.080093 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:51.081629 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.083850 (+ 2221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.088943 (+ 5093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.090712 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.090833 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.091114 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.091811 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.092015 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":2829,"threads_started":1}
I20260808 13:10:51.098217 7369 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a9f18f61647b88c209c2a8367d619 (DEFAULT_TABLE table=test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:51.100704 7369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a9f18f61647b88c209c2a8367d619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:51.102574 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.076718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44869 (local address 127.0.0.1:43194)
0808 13:10:51.088960 (+ 12242us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:51.089012 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:51.089198 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:51.090466 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:51.090472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:51.090500 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:51.091885 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.091903 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.094688 (+ 2785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.094695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.098337 (+ 3642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.098350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.101399 (+ 3049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.101409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.101644 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.101679 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.101792 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.101932 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12058,"thread_start_us":10558,"threads_started":1}
I20260808 13:10:51.102640 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.078972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.31.59.129:44869)
0808 13:10:51.081439 (+ 2467us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.081472 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.089344 (+ 7872us) server_negotiation.cc:407] Connection header received
0808 13:10:51.089509 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.089518 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.089846 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.090087 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:51.092283 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.094414 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.098522 (+ 4108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.101022 (+ 2500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.101202 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.101482 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.101850 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.102117 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":1364,"threads_started":1}
I20260808 13:10:51.104010 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.076535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43315 (local address 127.0.0.1:58878)
0808 13:10:51.089189 (+ 12654us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:51.089207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:51.089354 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:51.090496 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:51.090500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:51.090518 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:51.092049 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.092063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.095195 (+ 3132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.095204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.099119 (+ 3915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.099135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.102762 (+ 3627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.102772 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.103052 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.103079 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.103165 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.103302 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12143,"thread_start_us":1054,"threads_started":1}
I20260808 13:10:51.104833 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.076879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.31.59.130:43315)
0808 13:10:51.087186 (+ 10307us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.087194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.089499 (+ 2305us) server_negotiation.cc:407] Connection header received
0808 13:10:51.089605 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.089611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.089882 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.090087 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:51.092259 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.095045 (+ 2786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.099333 (+ 4288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.101355 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.101555 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.102393 (+ 838us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.104328 (+ 1935us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.104478 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10060,"thread_start_us":9913,"threads_started":1}
I20260808 13:10:51.107933 7061 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a9f18f61647b88c209c2a8367d619 (DEFAULT_TABLE table=test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:51.108273 7220 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a9f18f61647b88c209c2a8367d619 (DEFAULT_TABLE table=test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:51.109964 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a9f18f61647b88c209c2a8367d619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:51.110949 7061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a9f18f61647b88c209c2a8367d619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:51.125336 7476 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Bootstrap starting.
I20260808 13:10:51.130187 7477 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Bootstrap starting.
I20260808 13:10:51.133769 7476 tablet_bootstrap.cc:654] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:51.135142 7478 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Bootstrap starting.
I20260808 13:10:51.136586 7476 log.cc:824] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:51.137001 7477 tablet_bootstrap.cc:654] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:51.139554 7477 log.cc:824] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:51.142541 7478 tablet_bootstrap.cc:654] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:51.145107 7478 log.cc:824] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:51.150738 7476 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: No bootstrap required, opened a new log
I20260808 13:10:51.151337 7476 ts_tablet_manager.cc:1397] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.014s
I20260808 13:10:51.156131 7478 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: No bootstrap required, opened a new log
I20260808 13:10:51.156574 7478 ts_tablet_manager.cc:1397] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.009s
I20260808 13:10:51.161513 7477 tablet_bootstrap.cc:492] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: No bootstrap required, opened a new log
I20260808 13:10:51.161895 7477 ts_tablet_manager.cc:1397] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.023s
I20260808 13:10:51.171417 7476 raft_consensus.cc:348] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.172098 7476 raft_consensus.cc:374] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:51.172339 7476 raft_consensus.cc:729] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7dd9c99c574d0489816e0505962ff8, State: Initialized, Role: FOLLOWER
I20260808 13:10:51.173197 7476 consensus_queue.cc:260] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.173832 7478 raft_consensus.cc:348] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.174609 7478 raft_consensus.cc:374] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:51.174835 7478 raft_consensus.cc:729] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f04941a99e4fe38adba3e29fbdbf18, State: Initialized, Role: FOLLOWER
I20260808 13:10:51.174696 7477 raft_consensus.cc:348] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.175429 7477 raft_consensus.cc:374] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:51.175650 7477 raft_consensus.cc:729] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591352facb7143999b896ae541d3f2c5, State: Initialized, Role: FOLLOWER
I20260808 13:10:51.175959 7441 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
I20260808 13:10:51.175679 7478 consensus_queue.cc:260] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.176565 7477 consensus_queue.cc:260] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.177109 7476 ts_tablet_manager.cc:1428] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Time spent starting tablet: real 0.025s user 0.015s sys 0.007s
I20260808 13:10:51.186560 7478 ts_tablet_manager.cc:1428] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Time spent starting tablet: real 0.030s user 0.023s sys 0.009s
I20260808 13:10:51.191421 7289 heartbeater.cc:503] Master 127.31.59.190:34673 was elected leader, sending a full tablet report...
I20260808 13:10:51.192741 7477 ts_tablet_manager.cc:1428] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Time spent starting tablet: real 0.031s user 0.006s sys 0.011s
W20260808 13:10:51.229017 7443 tablet.cc:2369] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:51.323204 7485 raft_consensus.cc:482] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:51.323946 7485 raft_consensus.cc:504] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.327116 7485 leader_election.cc:288] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791), c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:51.339526 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.328131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:49921 (local address 127.31.59.129:44869)
0808 13:10:51.328472 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.328479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.330497 (+ 2018us) server_negotiation.cc:407] Connection header received
0808 13:10:51.330696 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.330700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.330764 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.330836 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:51.332098 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.333292 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.336635 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.338182 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.338375 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.338460 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.338996 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.339128 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:10:51.339670 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.327458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:58485 (local address 127.31.59.131:45791)
0808 13:10:51.327670 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.327676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.330093 (+ 2417us) server_negotiation.cc:407] Connection header received
0808 13:10:51.330341 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.330348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.330414 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.330489 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:51.331708 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.333154 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.336681 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.338296 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.338479 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.338558 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.339157 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.339291 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:10:51.339833 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.328195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44869 (local address 127.31.59.130:49921)
0808 13:10:51.330392 (+ 2197us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:51.330406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:51.330548 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:51.330874 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:51.330878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:51.330904 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:51.331834 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.331845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.333517 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.333522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.336481 (+ 2959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.336496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.338639 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.338646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.338832 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.338857 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.338945 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.339110 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2054,"thread_start_us":1649,"threads_started":1}
I20260808 13:10:51.340505 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.327144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:45791 (local address 127.31.59.130:58485)
0808 13:10:51.329967 (+ 2823us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:51.329983 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:51.330200 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:51.330583 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:51.330587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:51.330616 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:51.331561 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.331575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.333279 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.333284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.336483 (+ 3199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:51.336497 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.338734 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.338740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:51.338952 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.338977 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:10:51.339118 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 13:10:51.339252 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2623,"mutex_wait_us":2,"thread_start_us":1997,"threads_started":1}
I20260808 13:10:51.341557 7083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a9f18f61647b88c209c2a8367d619" candidate_uuid: "591352facb7143999b896ae541d3f2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" is_pre_election: true
I20260808 13:10:51.342129 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a9f18f61647b88c209c2a8367d619" candidate_uuid: "591352facb7143999b896ae541d3f2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7dd9c99c574d0489816e0505962ff8" is_pre_election: true
I20260808 13:10:51.342626 7083 raft_consensus.cc:2454] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 591352facb7143999b896ae541d3f2c5 in term 0.
I20260808 13:10:51.343358 7390 raft_consensus.cc:2454] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 591352facb7143999b896ae541d3f2c5 in term 0.
I20260808 13:10:51.343816 7170 leader_election.cc:302] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591352facb7143999b896ae541d3f2c5, c6f04941a99e4fe38adba3e29fbdbf18; no voters:
I20260808 13:10:51.344599 7485 raft_consensus.cc:2781] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:51.344846 7485 raft_consensus.cc:482] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:51.345031 7485 raft_consensus.cc:3037] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:51.349221 7485 raft_consensus.cc:504] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.350257 7485 leader_election.cc:288] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [CANDIDATE]: Term 1 election: Requested vote from peers 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791), c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869)
I20260808 13:10:51.350704 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a9f18f61647b88c209c2a8367d619" candidate_uuid: "591352facb7143999b896ae541d3f2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7dd9c99c574d0489816e0505962ff8"
I20260808 13:10:51.350929 7083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a9f18f61647b88c209c2a8367d619" candidate_uuid: "591352facb7143999b896ae541d3f2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f04941a99e4fe38adba3e29fbdbf18"
I20260808 13:10:51.350984 7390 raft_consensus.cc:3037] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:51.351212 7083 raft_consensus.cc:3037] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:51.354967 7390 raft_consensus.cc:2454] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591352facb7143999b896ae541d3f2c5 in term 1.
I20260808 13:10:51.355285 7083 raft_consensus.cc:2454] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591352facb7143999b896ae541d3f2c5 in term 1.
I20260808 13:10:51.355599 7169 leader_election.cc:302] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591352facb7143999b896ae541d3f2c5, 8c7dd9c99c574d0489816e0505962ff8; no voters:
I20260808 13:10:51.356123 7485 raft_consensus.cc:2781] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:51.357216 7485 raft_consensus.cc:686] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 1 LEADER]: Becoming Leader. State: Replica: 591352facb7143999b896ae541d3f2c5, State: Running, Role: LEADER
I20260808 13:10:51.357928 7485 consensus_queue.cc:237] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:51.364800 6090 catalog_manager.cc:5495] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 591352facb7143999b896ae541d3f2c5 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "591352facb7143999b896ae541d3f2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } health_report { overall_health: HEALTHY } } }
W20260808 13:10:51.395901 7294 tablet.cc:2369] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:51.402180 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.394198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.31.59.130:43315)
0808 13:10:51.394431 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.394436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.394852 (+ 416us) server_negotiation.cc:407] Connection header received
0808 13:10:51.395078 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.395083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.395175 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.395334 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:51.395949 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.397245 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.399548 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.400298 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.400413 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.400975 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:51.401501 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:51.401654 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.401709 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.401790 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 13:10:51.411420 7137 tablet.cc:2369] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:51.413656 7390 raft_consensus.cc:1261] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Refusing update from remote peer 591352facb7143999b896ae541d3f2c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:51.413645 7083 raft_consensus.cc:1261] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Refusing update from remote peer 591352facb7143999b896ae541d3f2c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:51.414855 7488 consensus_queue.cc:1035] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:10:51.415634 7485 consensus_queue.cc:1035] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:51.433696 7491 mvcc.cc:205] Tried to move back new op lower bound from 7316253292178292736 to 7316253291966971904. Current Snapshot: MvccSnapshot[applied={T|T < 7316253292178292736}]
I20260808 13:10:51.434352 7492 mvcc.cc:205] Tried to move back new op lower bound from 7316253292178292736 to 7316253291966971904. Current Snapshot: MvccSnapshot[applied={T|T < 7316253292178292736}]
I20260808 13:10:51.458184 6091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:51.458776 6091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:51.474763 6091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d82adb8c9a974543837b6ff639eae37c: Sending DeleteTablet for 3 replicas of tablet ca0a9f18f61647b88c209c2a8367d619
I20260808 13:10:51.476478 7061 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0a9f18f61647b88c209c2a8367d619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:51 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:43194
I20260808 13:10:51.476692 7369 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0a9f18f61647b88c209c2a8367d619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:51 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:37068
I20260808 13:10:51.478058 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0a9f18f61647b88c209c2a8367d619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:51 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:58878
I20260808 13:10:51.478948 7503 tablet_replica.cc:330] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: stopping tablet replica
I20260808 13:10:51.479537 7503 raft_consensus.cc:2229] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:51.480546 7503 raft_consensus.cc:2258] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:51.486690 7503 ts_tablet_manager.cc:1905] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:51.494392 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6056
I20260808 13:10:51.496189 6124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:51.502270 7503 ts_tablet_manager.cc:1918] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:51.502637 7503 log.cc:1196] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/ca0a9f18f61647b88c209c2a8367d619
I20260808 13:10:51.503621 7503 ts_tablet_manager.cc:1939] T ca0a9f18f61647b88c209c2a8367d619 P 8c7dd9c99c574d0489816e0505962ff8: Deleting consensus metadata
I20260808 13:10:51.510150 6075 catalog_manager.cc:4841] TS 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791): tablet ca0a9f18f61647b88c209c2a8367d619 (table test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]) successfully deleted
I20260808 13:10:51.515913 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.479459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.31.59.189:33935)
0808 13:10:51.479656 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.479663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.492493 (+ 12830us) server_negotiation.cc:407] Connection header received
0808 13:10:51.495781 (+ 3288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.495787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.495882 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.495964 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.498547 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.499669 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.501576 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.502179 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.502286 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.505974 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.505980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.505983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.506020 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.510802 (+ 4782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.512234 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.512241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.512244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.512437 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.513024 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.513030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.513033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.513504 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.513818 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.514649 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.514995 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:10:51.533627 7505 tablet_replica.cc:330] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: stopping tablet replica
I20260808 13:10:51.536425 7502 tablet_replica.cc:330] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: stopping tablet replica
I20260808 13:10:51.537391 7505 raft_consensus.cc:2229] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:51.538028 7502 raft_consensus.cc:2229] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:51.538895 7505 raft_consensus.cc:2258] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:51.539108 7502 raft_consensus.cc:2258] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:51.543502 7505 ts_tablet_manager.cc:1905] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:51.544497 7502 ts_tablet_manager.cc:1905] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:51.554194 7505 ts_tablet_manager.cc:1918] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:51.554721 7505 log.cc:1196] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/ca0a9f18f61647b88c209c2a8367d619
I20260808 13:10:51.554819 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.493097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.31.59.188:35963)
0808 13:10:51.493281 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.493286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.515505 (+ 22219us) server_negotiation.cc:407] Connection header received
0808 13:10:51.515877 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.515884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.515979 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.516072 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.519456 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.521328 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.524207 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.525275 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.525376 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.531723 (+ 6347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.531732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.531737 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.531795 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.538384 (+ 6589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.539831 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.539837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.539841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.541276 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.542697 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.542702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.542705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.543109 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.543373 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.553810 (+ 10437us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.554163 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:10:51.556223 7505 ts_tablet_manager.cc:1939] T ca0a9f18f61647b88c209c2a8367d619 P 591352facb7143999b896ae541d3f2c5: Deleting consensus metadata
I20260808 13:10:51.557245 7502 ts_tablet_manager.cc:1918] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:51.557582 7502 log.cc:1196] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/ca0a9f18f61647b88c209c2a8367d619
I20260808 13:10:51.558583 7502 ts_tablet_manager.cc:1939] T ca0a9f18f61647b88c209c2a8367d619 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting consensus metadata
I20260808 13:10:51.729902 6076 catalog_manager.cc:4841] TS c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869): tablet ca0a9f18f61647b88c209c2a8367d619 (table test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]) successfully deleted
I20260808 13:10:51.729979 6077 catalog_manager.cc:4841] TS 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315): tablet ca0a9f18f61647b88c209c2a8367d619 (table test-table [id=ebc7a3d9d52b4c048696b59f1a1f8c8d]) successfully deleted
I20260808 13:10:51.731403 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6056
I20260808 13:10:51.781955 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.764160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.31.59.188:35963)
0808 13:10:51.764405 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.764412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.764521 (+ 109us) server_negotiation.cc:407] Connection header received
0808 13:10:51.764727 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.764732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.764797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.764877 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.768723 (+ 3846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.769603 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.771312 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.771776 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.771872 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.775168 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.775176 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.775180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.775225 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.778628 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.779448 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.779454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.779457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.779611 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.780015 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.780021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.780024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.780386 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.780682 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.781314 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.781586 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:10:51.783434 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:51.762691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.31.59.189:33935)
0808 13:10:51.762949 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:51.762956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:51.762984 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:10:51.763281 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:51.763288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:51.763378 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:51.763454 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:51.766090 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.767306 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.769441 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:51.770485 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:51.770610 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:51.774278 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:51.774284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:51.774287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:51.774323 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:51.779368 (+ 5045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.780233 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.780239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.780243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.780439 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:51.780947 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:51.780952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:51.780955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:51.781354 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:51.781631 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:51.782858 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0808 13:10:51.783117 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:10:52.447826 7514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.446968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:33105)
0808 13:10:52.447446 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.447599 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
W20260808 13:10:52.448418 7134 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:34673 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.479622 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.478856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:36147)
0808 13:10:52.479305 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.479430 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":218,"threads_started":1}
W20260808 13:10:52.480024 7289 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:34673 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.481294 7516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.480596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:46987)
0808 13:10:52.481004 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.481120 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":210,"threads_started":1}
W20260808 13:10:52.481675 7441 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:34673 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.974346 7517 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d82adb8c9a974543837b6ff639eae37c)
I20260808 13:10:52.974489 7518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d82adb8c9a974543837b6ff639eae37c)
I20260808 13:10:52.974704 7518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } }
I20260808 13:10:52.974655 7517 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } }
I20260808 13:10:52.975594 7518 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673), c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189:33935)
I20260808 13:10:52.976272 7517 leader_election.cc:288] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673), 1f3bfccf7f11421694a2cbd57f1ddae6 (127.31.59.188:35963)
I20260808 13:10:52.976897 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.975680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55150)
0808 13:10:52.976333 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.976622 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":369,"threads_started":1}
W20260808 13:10:52.977914 6926 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.978909 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.976699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55160)
0808 13:10:52.978422 (+ 1723us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.978623 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1591,"thread_start_us":1436,"threads_started":1}
I20260808 13:10:52.981168 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.980543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55162)
0808 13:10:52.980769 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.980942 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260808 13:10:52.981596 6926 leader_election.cc:334] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673): Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
W20260808 13:10:52.983882 6722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.984881 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.976468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:33935 (local address 127.0.0.1:55110)
0808 13:10:52.977187 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.977206 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:52.977342 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:52.978524 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:52.978531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:52.978562 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:52.978832 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:52.978843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.980547 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.980553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:52.982588 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:52.982606 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.984094 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.984100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:52.984290 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:52.984328 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:10:52.984428 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 13:10:52.984606 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":1116,"threads_started":1}
I20260808 13:10:52.985419 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.976791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.31.59.189:33935)
0808 13:10:52.978147 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:52.978153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:52.978174 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:10:52.978248 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:52.978257 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:52.978343 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:52.978968 (+ 625us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:52.979229 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.980375 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.982781 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.983920 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.984047 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:52.984153 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:52.984770 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 13:10:52.984935 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":1552,"threads_started":1}
I20260808 13:10:52.986129 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.985464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55168)
0808 13:10:52.985646 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.985877 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:10:52.986168 6749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" is_pre_election: true
W20260808 13:10:52.986361 6722 leader_election.cc:334] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673): Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.986467 6749 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f3bfccf7f11421694a2cbd57f1ddae6 in term 1.
I20260808 13:10:52.986976 6926 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f3bfccf7f11421694a2cbd57f1ddae6, c86d96cbfd324ad1ac8d9c4acfae8471; no voters: d82adb8c9a974543837b6ff639eae37c
I20260808 13:10:52.987260 7518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:10:52.987396 7518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d82adb8c9a974543837b6ff639eae37c)
I20260808 13:10:52.987521 7518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:10:52.987602 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.978689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35963 (local address 127.0.0.1:39518)
0808 13:10:52.979141 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.979160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:52.979321 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:52.980926 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:52.980932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:52.980961 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:52.981288 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:52.981299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.983390 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.983396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:52.985356 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:52.985368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.986909 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.986914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:52.987039 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:52.987062 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:10:52.987159 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:10:52.987307 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":1,"thread_start_us":4838,"threads_started":1}
I20260808 13:10:52.987788 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.979802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39518 (local address 127.31.59.188:35963)
0808 13:10:52.980545 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:52.980551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:52.980576 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:10:52.980642 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:52.980651 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:52.980725 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:52.981430 (+ 705us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:52.981879 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.983246 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.985528 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:52.986738 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:52.986870 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:52.986977 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:52.987201 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:10:52.987349 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":462,"threads_started":1}
I20260808 13:10:52.988358 6952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" is_pre_election: true
I20260808 13:10:52.989884 7518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } }
I20260808 13:10:52.990296 6952 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c86d96cbfd324ad1ac8d9c4acfae8471 in current term 2: Already voted for candidate 1f3bfccf7f11421694a2cbd57f1ddae6 in this term.
I20260808 13:10:52.990471 7518 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 election: Requested vote from peers d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673), c86d96cbfd324ad1ac8d9c4acfae8471 (127.31.59.189:33935)
I20260808 13:10:52.990940 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.990577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55178)
0808 13:10:52.990704 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.990779 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 13:10:52.991214 6749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471"
I20260808 13:10:52.991430 6749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:10:52.991853 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:52.991520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55190)
0808 13:10:52.991645 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:52.991755 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 13:10:52.992014 6926 leader_election.cc:334] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673): Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
I20260808 13:10:52.992033 6722 leader_election.cc:302] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c86d96cbfd324ad1ac8d9c4acfae8471; no voters: 1f3bfccf7f11421694a2cbd57f1ddae6, d82adb8c9a974543837b6ff639eae37c
I20260808 13:10:52.993975 6749 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f3bfccf7f11421694a2cbd57f1ddae6 in term 2.
I20260808 13:10:52.994138 7517 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:10:52.994410 6926 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f3bfccf7f11421694a2cbd57f1ddae6, c86d96cbfd324ad1ac8d9c4acfae8471; no voters: d82adb8c9a974543837b6ff639eae37c
I20260808 13:10:52.994700 7518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:10:52.995137 7518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [term 2 LEADER]: Becoming Leader. State: Replica: 1f3bfccf7f11421694a2cbd57f1ddae6, State: Running, Role: LEADER
I20260808 13:10:52.995532 7518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } }
I20260808 13:10:52.997378 7527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f3bfccf7f11421694a2cbd57f1ddae6. Latest consensus state: current_term: 2 leader_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:52.997852 7527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:52.998483 7529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:53.002097 7529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:53.002875 7529 catalog_manager.cc:1259] Loaded cluster ID: 7a6fb6690c1d4b8abba1ec2d23a54360
I20260808 13:10:53.002940 7529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:53.004005 7529 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:53.004772 7529 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Loaded TSK: 0
I20260808 13:10:53.005581 7529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:10:53.024310 6942 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:53.031525 6942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:53.035624 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.035235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55200)
0808 13:10:53.035393 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.035478 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 13:10:53.036054 6749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [term 2 FOLLOWER]: Refusing update from remote peer 1f3bfccf7f11421694a2cbd57f1ddae6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 13:10:53.036643 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.036318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55206)
0808 13:10:53.036449 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.036520 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 13:10:53.036824 7527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 13:10:53.037077 6926 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 -> Peer d82adb8c9a974543837b6ff639eae37c (127.31.59.190:34673): Couldn't send request to peer d82adb8c9a974543837b6ff639eae37c. Status: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:10:53.039578 7517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f3bfccf7f11421694a2cbd57f1ddae6. Latest consensus state: current_term: 2 leader_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:53.039829 7517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:53.041172 7518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:53.041599 7518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:53.041709 7517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d82adb8c9a974543837b6ff639eae37c" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34673 } } peers { permanent_uuid: "c86d96cbfd324ad1ac8d9c4acfae8471" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 33935 } attrs { promote: false } } peers { permanent_uuid: "1f3bfccf7f11421694a2cbd57f1ddae6" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35963 } attrs { promote: false } } }
I20260808 13:10:53.041982 7517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d96cbfd324ad1ac8d9c4acfae8471 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:10:53.058377 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.049921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:45791 (local address 127.0.0.1:37078)
0808 13:10:53.050068 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.050086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:53.050213 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:53.051389 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:53.051394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:53.051422 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:53.051858 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.051871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.053549 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.053556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.055722 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.055737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.057478 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.057485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.057665 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.057699 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:10:53.057813 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 13:10:53.057971 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 13:10:53.058439 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.050343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.31.59.131:45791)
0808 13:10:53.050871 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.050876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.050896 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:53.050983 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.050988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.051061 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.051565 (+ 504us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:53.052184 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.053393 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.055931 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.057239 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.057382 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.057528 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.057859 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.058067 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":283,"threads_started":1}
I20260808 13:10:53.059508 7369 tablet_service.cc:1467] Processing CreateTablet for tablet 9354decac8124a0ea91bb1115c112e87 (DEFAULT_TABLE table=test-table [id=18139fccac9c456791227d4b5cfabf91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:53.060011 7369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9354decac8124a0ea91bb1115c112e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:53.063591 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.051293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.31.59.130:43315)
0808 13:10:53.057003 (+ 5710us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.057007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.057022 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:10:53.057080 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.057083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.057138 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.057393 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:53.058360 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.059320 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.061537 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.062649 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.062773 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.062910 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.063176 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.063320 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5607,"thread_start_us":5503,"threads_started":1}
I20260808 13:10:53.063617 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.050805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43315 (local address 127.0.0.1:58900)
0808 13:10:53.051470 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.051498 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:53.051804 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:53.057661 (+ 5857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:53.057666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:53.057691 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:53.058153 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.058165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.059441 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.059446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.061348 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.061362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.062855 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.062863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.063023 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.063054 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:10:53.063132 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:10:53.063274 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":609,"threads_started":1}
I20260808 13:10:53.064468 7220 tablet_service.cc:1467] Processing CreateTablet for tablet 9354decac8124a0ea91bb1115c112e87 (DEFAULT_TABLE table=test-table [id=18139fccac9c456791227d4b5cfabf91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:53.064925 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9354decac8124a0ea91bb1115c112e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:53.070159 7542 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Bootstrap starting.
I20260808 13:10:53.071447 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.049525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44869 (local address 127.0.0.1:43198)
0808 13:10:53.049802 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.049821 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:53.049927 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:53.060658 (+ 10731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:53.060663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:53.060688 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:53.060861 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.060872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.062242 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.062254 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.064034 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.064049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.070746 (+ 6697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.070753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.070945 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.070979 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:10:53.071073 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 13:10:53.071207 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 13:10:53.072362 7542 tablet_bootstrap.cc:654] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:53.073313 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.049694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.31.59.129:44869)
0808 13:10:53.059923 (+ 10229us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.059930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.059954 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:10:53.060028 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.060033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.060112 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.060995 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:53.061130 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.062086 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.069668 (+ 7582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.070546 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.070702 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.070844 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.072908 (+ 2064us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.073072 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9903,"thread_start_us":12633,"threads_started":1}
I20260808 13:10:53.074050 7061 tablet_service.cc:1467] Processing CreateTablet for tablet 9354decac8124a0ea91bb1115c112e87 (DEFAULT_TABLE table=test-table [id=18139fccac9c456791227d4b5cfabf91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:53.074461 7061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9354decac8124a0ea91bb1115c112e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:53.074705 7541 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Bootstrap starting.
I20260808 13:10:53.076601 7541 tablet_bootstrap.cc:654] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:53.079088 7542 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: No bootstrap required, opened a new log
I20260808 13:10:53.079334 7542 ts_tablet_manager.cc:1397] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260808 13:10:53.080219 7542 raft_consensus.cc:348] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.080494 7542 raft_consensus.cc:374] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:53.080616 7542 raft_consensus.cc:729] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591352facb7143999b896ae541d3f2c5, State: Initialized, Role: FOLLOWER
I20260808 13:10:53.080857 7542 consensus_queue.cc:260] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.081809 7542 ts_tablet_manager.cc:1428] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 13:10:53.082626 7547 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Bootstrap starting.
I20260808 13:10:53.084645 7547 tablet_bootstrap.cc:654] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:53.085819 7288 heartbeater.cc:503] Master 127.31.59.188:35963 was elected leader, sending a full tablet report...
I20260808 13:10:53.086439 7541 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: No bootstrap required, opened a new log
I20260808 13:10:53.086645 7541 ts_tablet_manager.cc:1397] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260808 13:10:53.086997 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.082628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:40393)
0808 13:10:53.086332 (+ 3704us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.086615 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2735,"thread_start_us":2595,"threads_started":1}
I20260808 13:10:53.087533 7541 raft_consensus.cc:348] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.088223 7541 raft_consensus.cc:374] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:53.088397 7541 raft_consensus.cc:729] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7dd9c99c574d0489816e0505962ff8, State: Initialized, Role: FOLLOWER
I20260808 13:10:53.088692 7541 consensus_queue.cc:260] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.089614 7541 ts_tablet_manager.cc:1428] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:53.093077 7440 heartbeater.cc:503] Master 127.31.59.188:35963 was elected leader, sending a full tablet report...
I20260808 13:10:53.093223 7547 tablet_bootstrap.cc:492] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: No bootstrap required, opened a new log
I20260808 13:10:53.093346 7547 ts_tablet_manager.cc:1397] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260808 13:10:53.094228 7547 raft_consensus.cc:348] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.094499 7547 raft_consensus.cc:374] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:53.094601 7547 raft_consensus.cc:729] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f04941a99e4fe38adba3e29fbdbf18, State: Initialized, Role: FOLLOWER
I20260808 13:10:53.094868 7547 consensus_queue.cc:260] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.095846 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.092146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:37117)
0808 13:10:53.095425 (+ 3279us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.095601 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3126,"thread_start_us":470,"threads_started":1}
I20260808 13:10:53.096530 7547 ts_tablet_manager.cc:1428] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:53.097405 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.096298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:40743)
0808 13:10:53.096949 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.097118 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":357,"threads_started":1}
I20260808 13:10:53.097605 7133 heartbeater.cc:503] Master 127.31.59.188:35963 was elected leader, sending a full tablet report...
I20260808 13:10:53.098448 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.098068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:50149)
0808 13:10:53.098223 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.098312 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 13:10:53.352880 7553 raft_consensus.cc:482] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:53.353125 7553 raft_consensus.cc:504] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.354779 7553 leader_election.cc:288] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791), 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
I20260808 13:10:53.361155 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.355133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:52855 (local address 127.31.59.131:45791)
0808 13:10:53.355422 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.355429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.355842 (+ 413us) server_negotiation.cc:407] Connection header received
0808 13:10:53.356053 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.356058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.356121 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.356183 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:53.356685 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.357440 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.359521 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.360260 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.360386 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.360456 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.360776 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.360905 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 13:10:53.361156 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.355289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:60393 (local address 127.31.59.130:43315)
0808 13:10:53.355501 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.355507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.355900 (+ 393us) server_negotiation.cc:407] Connection header received
0808 13:10:53.356059 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.356064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.356144 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.356228 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:53.356830 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.357625 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.359494 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.360295 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.360457 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.360528 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.360812 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.360935 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:10:53.361176 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.355003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:45791 (local address 127.31.59.129:52855)
0808 13:10:53.355574 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.355594 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:53.355905 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:53.356229 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:53.356234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:53.356259 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:53.356547 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.356558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.357602 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.357607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.359377 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.359389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.360439 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.360445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.360630 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.360654 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:10:53.360725 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 13:10:53.360853 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":55}
I20260808 13:10:53.361354 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.355116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43315 (local address 127.31.59.129:60393)
0808 13:10:53.355781 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.355795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:53.355933 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:53.356257 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:53.356262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:53.356312 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:53.356671 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.356682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.357742 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.357752 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.359351 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:53.359363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.360523 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.360528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:53.360651 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.360673 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:10:53.360784 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 13:10:53.360905 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":22,"spinlock_wait_cycles":7040,"thread_start_us":312,"threads_started":1}
I20260808 13:10:53.361997 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9354decac8124a0ea91bb1115c112e87" candidate_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7dd9c99c574d0489816e0505962ff8" is_pre_election: true
I20260808 13:10:53.362215 7390 raft_consensus.cc:2454] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f04941a99e4fe38adba3e29fbdbf18 in term 0.
I20260808 13:10:53.362190 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9354decac8124a0ea91bb1115c112e87" candidate_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591352facb7143999b896ae541d3f2c5" is_pre_election: true
I20260808 13:10:53.362592 7241 raft_consensus.cc:2454] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f04941a99e4fe38adba3e29fbdbf18 in term 0.
I20260808 13:10:53.362670 7013 leader_election.cc:302] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c7dd9c99c574d0489816e0505962ff8, c6f04941a99e4fe38adba3e29fbdbf18; no voters:
I20260808 13:10:53.363081 7553 raft_consensus.cc:2781] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:53.363212 7553 raft_consensus.cc:482] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:53.363312 7553 raft_consensus.cc:3037] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:53.365424 7553 raft_consensus.cc:504] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.366055 7553 leader_election.cc:288] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [CANDIDATE]: Term 1 election: Requested vote from peers 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791), 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315)
I20260808 13:10:53.366451 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9354decac8124a0ea91bb1115c112e87" candidate_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7dd9c99c574d0489816e0505962ff8"
I20260808 13:10:53.366509 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9354decac8124a0ea91bb1115c112e87" candidate_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591352facb7143999b896ae541d3f2c5"
I20260808 13:10:53.366688 7390 raft_consensus.cc:3037] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:53.366688 7241 raft_consensus.cc:3037] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:53.369659 7390 raft_consensus.cc:2454] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f04941a99e4fe38adba3e29fbdbf18 in term 1.
I20260808 13:10:53.369658 7241 raft_consensus.cc:2454] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f04941a99e4fe38adba3e29fbdbf18 in term 1.
I20260808 13:10:53.370035 7015 leader_election.cc:302] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591352facb7143999b896ae541d3f2c5, c6f04941a99e4fe38adba3e29fbdbf18; no voters:
I20260808 13:10:53.370344 7553 raft_consensus.cc:2781] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:53.370788 7553 raft_consensus.cc:686] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 LEADER]: Becoming Leader. State: Replica: c6f04941a99e4fe38adba3e29fbdbf18, State: Running, Role: LEADER
I20260808 13:10:53.371333 7553 consensus_queue.cc:237] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } }
I20260808 13:10:53.371847 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.371397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:60593)
0808 13:10:53.371563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.371679 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 13:10:53.374986 6942 catalog_manager.cc:5495] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f04941a99e4fe38adba3e29fbdbf18" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:53.425026 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.420401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.31.59.129:44869)
0808 13:10:53.420609 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.420614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.420630 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:10:53.420718 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.420721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.420801 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.420922 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:53.421216 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.421967 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.423129 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.423559 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.423643 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.424007 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:53.424519 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:53.424618 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.424705 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.424768 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:10:53.429116 7390 raft_consensus.cc:1261] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Refusing update from remote peer c6f04941a99e4fe38adba3e29fbdbf18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:53.429396 7241 raft_consensus.cc:1261] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Refusing update from remote peer c6f04941a99e4fe38adba3e29fbdbf18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:53.429857 7556 consensus_queue.cc:1035] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7dd9c99c574d0489816e0505962ff8" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:53.430275 7553 consensus_queue.cc:1035] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "591352facb7143999b896ae541d3f2c5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:53.432601 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.432104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:37903)
0808 13:10:53.432304 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.432423 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:10:53.434254 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.433752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:45541)
0808 13:10:53.433936 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.434048 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 13:10:53.438596 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.438064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.130:37345)
0808 13:10:53.438278 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.438408 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 13:10:53.443800 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.443251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:45031)
0808 13:10:53.443512 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.443608 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 13:10:53.449193 7560 mvcc.cc:205] Tried to move back new op lower bound from 7316253300438990848 to 7316253300213137408. Current Snapshot: MvccSnapshot[applied={T|T < 7316253300438990848}]
I20260808 13:10:53.450021 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.449477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.131:47399)
0808 13:10:53.449720 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.449846 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 13:10:53.451617 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.451233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.31.59.129:35479)
0808 13:10:53.451387 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.451466 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 13:10:53.459156 6941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:53.459476 6941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:53.465387 6941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f3bfccf7f11421694a2cbd57f1ddae6: Sending DeleteTablet for 3 replicas of tablet 9354decac8124a0ea91bb1115c112e87
I20260808 13:10:53.466584 7061 tablet_service.cc:1514] Processing DeleteTablet for tablet 9354decac8124a0ea91bb1115c112e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:53 UTC) from {username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:43198
I20260808 13:10:53.466658 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet 9354decac8124a0ea91bb1115c112e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:53 UTC) from {username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:58900
I20260808 13:10:53.467072 7369 tablet_service.cc:1514] Processing DeleteTablet for tablet 9354decac8124a0ea91bb1115c112e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:53 UTC) from {username='oryx', principal='oryx/127.31.59.188@KRBTEST.COM'} at 127.0.0.1:37078
I20260808 13:10:53.468547 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6913
I20260808 13:10:53.471239 7571 tablet_replica.cc:330] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: stopping tablet replica
I20260808 13:10:53.471549 7571 raft_consensus.cc:2229] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:53.471975 7571 raft_consensus.cc:2258] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:53.473266 7571 ts_tablet_manager.cc:1905] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:53.475350 7570 tablet_replica.cc:330] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: stopping tablet replica
I20260808 13:10:53.475584 7569 tablet_replica.cc:330] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: stopping tablet replica
I20260808 13:10:53.475855 7569 raft_consensus.cc:2229] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:53.476109 7570 raft_consensus.cc:2229] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:53.476540 7570 raft_consensus.cc:2258] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:53.476622 7569 raft_consensus.cc:2258] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:53.477748 7569 ts_tablet_manager.cc:1905] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:53.481550 7571 ts_tablet_manager.cc:1918] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:53.481947 7571 log.cc:1196] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/9354decac8124a0ea91bb1115c112e87
I20260808 13:10:53.482532 7571 ts_tablet_manager.cc:1939] T 9354decac8124a0ea91bb1115c112e87 P 8c7dd9c99c574d0489816e0505962ff8: Deleting consensus metadata
I20260808 13:10:53.484273 7570 ts_tablet_manager.cc:1905] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:53.485276 7569 ts_tablet_manager.cc:1918] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:53.485535 7569 log.cc:1196] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/9354decac8124a0ea91bb1115c112e87
I20260808 13:10:53.486075 7569 ts_tablet_manager.cc:1939] T 9354decac8124a0ea91bb1115c112e87 P c6f04941a99e4fe38adba3e29fbdbf18: Deleting consensus metadata
I20260808 13:10:53.487656 6927 catalog_manager.cc:4841] TS c6f04941a99e4fe38adba3e29fbdbf18 (127.31.59.129:44869): tablet 9354decac8124a0ea91bb1115c112e87 (table test-table [id=18139fccac9c456791227d4b5cfabf91]) successfully deleted
I20260808 13:10:53.490551 7570 ts_tablet_manager.cc:1918] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:53.490689 7570 log.cc:1196] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/9354decac8124a0ea91bb1115c112e87
I20260808 13:10:53.491056 7570 ts_tablet_manager.cc:1939] T 9354decac8124a0ea91bb1115c112e87 P 591352facb7143999b896ae541d3f2c5: Deleting consensus metadata
I20260808 13:10:53.492105 6928 catalog_manager.cc:4841] TS 591352facb7143999b896ae541d3f2c5 (127.31.59.130:43315): tablet 9354decac8124a0ea91bb1115c112e87 (table test-table [id=18139fccac9c456791227d4b5cfabf91]) successfully deleted
I20260808 13:10:53.492372 6926 catalog_manager.cc:4841] TS 8c7dd9c99c574d0489816e0505962ff8 (127.31.59.131:45791): tablet 9354decac8124a0ea91bb1115c112e87 (table test-table [id=18139fccac9c456791227d4b5cfabf91]) successfully deleted
I20260808 13:10:53.494596 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.469648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.31.59.129:44869)
0808 13:10:53.469844 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.469850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.475881 (+ 6031us) server_negotiation.cc:407] Connection header received
0808 13:10:53.476169 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.476175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.476276 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.476348 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:53.478387 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.479564 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.481440 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.482105 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.482221 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.486775 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:53.486792 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:53.486813 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:53.486857 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:53.491401 (+ 4544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.492320 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.492330 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.492334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.492521 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.492940 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.492944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.492946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.493351 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:53.493683 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.494012 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.494289 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:10:53.495301 7127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:53.592275 7574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.591575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55212)
0808 13:10:53.592009 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.592100 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":238,"threads_started":1}
I20260808 13:10:53.593577 7574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.593201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34673 (local address 127.0.0.1:55222)
0808 13:10:53.593347 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:53.593420 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:34673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 13:10:53.598775 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6913
I20260808 13:10:53.620784 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 7153
I20260808 13:10:53.633016 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.621538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.31.59.130:43315)
0808 13:10:53.621701 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.621706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.621789 (+ 83us) server_negotiation.cc:407] Connection header received
0808 13:10:53.621932 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.621936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.621999 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.622085 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:53.623272 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.624030 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.625260 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.625732 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.625829 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.628123 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:53.628140 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:53.628165 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:53.628196 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:53.630811 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.631434 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.631441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.631443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.631558 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.631817 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.631820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.631822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.632181 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:53.632407 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.632591 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.632759 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:10:53.633710 7283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:53.754861 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 7153
I20260808 13:10:53.778594 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 7301
I20260808 13:10:53.791275 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:53.779369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.31.59.131:45791)
0808 13:10:53.779539 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:53.779544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:53.779563 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:10:53.779707 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:53.779711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:53.779774 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:53.779863 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:53.781137 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.781949 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.783209 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:53.783629 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:53.783723 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:53.786280 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:53.786297 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:53.786314 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:53.786347 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:53.789023 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.789695 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.789702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.789705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.789828 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:53.790104 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:53.790107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:53.790109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:53.790467 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:53.790703 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:53.790865 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 13:10:53.791025 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:10:53.791963 7434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:53.885780 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 7301
I20260808 13:10:53.903884 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6591
I20260808 13:10:53.904982 6770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:54.026393 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6591
I20260808 13:10:54.047472 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 6701
I20260808 13:10:54.048547 6973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:10:54.176311 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 6701
2026-08-08T13:10:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10238 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:10:54 dist-test-slave-49h3 krb5kdc[7583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:10:54 dist-test-slave-49h3 krb5kdc[7583](info): set up 2 sockets
Aug 08 13:10:54 dist-test-slave-49h3 krb5kdc[7583](info): commencing operation
krb5kdc: starting...
W20260808 13:10:55.259634 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.008s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:10:55 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:10:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab.
Entry for principal oryx/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab.
I20260808 13:10:55.513309 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:43557
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab
--principal=oryx/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:43557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:55.847755 7596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:55.848078 7596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:55.848217 7596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:55.855561 7596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:10:55.855693 7596 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:10:55.855765 7596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:55.855856 7596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:55.855917 7596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:10:55.855999 7596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:10:55.866194 7596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:43557
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.190.keytab
--principal=oryx/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:55.867631 7596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:55.869033 7596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:55.880926 7601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:55.881049 7604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:55.881069 7602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:55.882115 7596 server_base.cc:1034] running on GCE node
I20260808 13:10:55.882875 7596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:55.884274 7596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:55.885522 7596 hybrid_clock.cc:648] HybridClock initialized: now 1786194655885417 us; error 44 us; skew 500 ppm
I20260808 13:10:55.885881 7596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:55 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194655, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:55.889123 7596 init.cc:377] Logged in from keytab as oryx/127.31.59.190@KRBTEST.COM (short username oryx)
I20260808 13:10:55.890647 7596 webserver.cc:460] Webserver started at http://127.31.59.190:42041/ using document root <none> and password file <none>
I20260808 13:10:55.891371 7596 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:55.891469 7596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:55.891798 7596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:55.894934 7596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "9f8ad272b3a241079ddb0c23be355a8e"
format_stamp: "Formatted at 2026-08-08 13:10:55 on dist-test-slave-49h3"
server_key: "04c75915b3b5242d5ff242fc20288904"
server_key_iv: "a337cd6ef236cedc383b49937ecf17ef"
server_key_version: "encryptionkey@0"
I20260808 13:10:55.895761 7596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "9f8ad272b3a241079ddb0c23be355a8e"
format_stamp: "Formatted at 2026-08-08 13:10:55 on dist-test-slave-49h3"
server_key: "04c75915b3b5242d5ff242fc20288904"
server_key_iv: "a337cd6ef236cedc383b49937ecf17ef"
server_key_version: "encryptionkey@0"
I20260808 13:10:55.900885 7596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:10:55.904373 7612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:55.905733 7596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:10:55.905895 7596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "9f8ad272b3a241079ddb0c23be355a8e"
format_stamp: "Formatted at 2026-08-08 13:10:55 on dist-test-slave-49h3"
server_key: "04c75915b3b5242d5ff242fc20288904"
server_key_iv: "a337cd6ef236cedc383b49937ecf17ef"
server_key_version: "encryptionkey@0"
I20260808 13:10:55.906111 7596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:55.924439 7596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:55.928889 7596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:55.929162 7596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:55.955307 7596 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:43557
I20260808 13:10:55.955328 7675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:43557 every 8 connection(s)
I20260808 13:10:55.956718 7596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:10:55.958257 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 7596
I20260808 13:10:55.958472 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:10:55.959110 31982 external_mini_cluster.cc:1428] Setting key 2eed733f999f0e0775d868d60a02a32e
I20260808 13:10:55.960924 7676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:10:55 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:55.974557 7676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Bootstrap starting.
I20260808 13:10:55.978709 7676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:55.979954 7676 log.cc:824] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:55.983320 7676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: No bootstrap required, opened a new log
I20260808 13:10:55.983479 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:55.961096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.31.59.190:43557)
0808 13:10:55.961945 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:55.961960 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:55.962013 (+ 53us) server_negotiation.cc:407] Connection header received
0808 13:10:55.964049 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:55.964079 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:55.964584 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:55.965053 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:55.966496 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:55.968544 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:55.969895 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:55.970950 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:55.971144 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:55.975211 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:55.975222 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:55.975241 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:55.975290 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:55.979074 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:55.979809 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:55.979818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:55.979826 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:55.980036 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:55.980411 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:55.980416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:55.980420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:55.981035 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:55.981277 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:55.981766 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 13:10:55.982193 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":350,"threads_started":1}
I20260808 13:10:55.991422 7676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } }
I20260808 13:10:55.991786 7676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:55.991864 7676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8ad272b3a241079ddb0c23be355a8e, State: Initialized, Role: FOLLOWER
I20260808 13:10:55.992357 7676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } }
I20260808 13:10:55.992564 7676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:10:55.992676 7676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:10:55.992805 7676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:55.995018 7676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } }
I20260808 13:10:55.995500 7676 leader_election.cc:302] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f8ad272b3a241079ddb0c23be355a8e; no voters:
I20260808 13:10:55.996023 7676 leader_election.cc:288] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:10:55.996170 7681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:55.996872 7681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [term 1 LEADER]: Becoming Leader. State: Replica: 9f8ad272b3a241079ddb0c23be355a8e, State: Running, Role: LEADER
I20260808 13:10:55.997409 7681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } }
I20260808 13:10:55.998283 7676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:10:56.000564 7682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f8ad272b3a241079ddb0c23be355a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } } }
I20260808 13:10:56.001003 7682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:56.000612 7683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f8ad272b3a241079ddb0c23be355a8e. Latest consensus state: current_term: 1 leader_uuid: "9f8ad272b3a241079ddb0c23be355a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8ad272b3a241079ddb0c23be355a8e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 43557 } } }
I20260808 13:10:56.001303 7683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e [sys.catalog]: This master's current role is: LEADER
I20260808 13:10:56.005645 7689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:10:56.009967 7689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:10:56.018029 7689 catalog_manager.cc:1347] Generated new cluster ID: 484a46f6985740c4a0e6f7613d7375b0
I20260808 13:10:56.018153 7689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:10:56.034384 7689 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:10:56.035693 7689 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:10:56.055420 7689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Generated new TSK 0
I20260808 13:10:56.056278 7689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
I20260808 13:10:56.122375 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:43557
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:56.461417 7702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:56.461730 7702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:56.461851 7702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:56.469184 7702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:56.469306 7702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:56.469400 7702 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:56.469517 7702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:10:56.479651 7702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:56.480902 7702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:56.482285 7702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:56.495522 7707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:56.495666 7708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:56.495645 7710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:56.496797 7702 server_base.cc:1034] running on GCE node
I20260808 13:10:56.497412 7702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:56.498428 7702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:56.499648 7702 hybrid_clock.cc:648] HybridClock initialized: now 1786194656499579 us; error 46 us; skew 500 ppm
I20260808 13:10:56.499943 7702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:56 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194656, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:56.503126 7702 init.cc:377] Logged in from keytab as oryx/127.31.59.129@KRBTEST.COM (short username oryx)
I20260808 13:10:56.504585 7702 webserver.cc:460] Webserver started at http://127.31.59.129:39635/ using document root <none> and password file <none>
I20260808 13:10:56.505262 7702 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:56.505380 7702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:56.505731 7702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:56.508689 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "da81fc3d2bff40f68b2ed1adeea6d329"
format_stamp: "Formatted at 2026-08-08 13:10:56 on dist-test-slave-49h3"
server_key: "d279a056779f4891cbaf31a58196337a"
server_key_iv: "6334433ba9b75b63b4b905a4bceef3c3"
server_key_version: "encryptionkey@0"
I20260808 13:10:56.509495 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "da81fc3d2bff40f68b2ed1adeea6d329"
format_stamp: "Formatted at 2026-08-08 13:10:56 on dist-test-slave-49h3"
server_key: "d279a056779f4891cbaf31a58196337a"
server_key_iv: "6334433ba9b75b63b4b905a4bceef3c3"
server_key_version: "encryptionkey@0"
I20260808 13:10:56.514900 7702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:10:56.518402 7718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:56.519670 7702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:10:56.519825 7702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "da81fc3d2bff40f68b2ed1adeea6d329"
format_stamp: "Formatted at 2026-08-08 13:10:56 on dist-test-slave-49h3"
server_key: "d279a056779f4891cbaf31a58196337a"
server_key_iv: "6334433ba9b75b63b4b905a4bceef3c3"
server_key_version: "encryptionkey@0"
I20260808 13:10:56.520032 7702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:56.534677 7702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:56.538615 7702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:56.538880 7702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:56.541680 7702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:56.541776 7702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:56.541917 7702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:56.541982 7702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:56 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194656, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:56.566551 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:56.542437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:46303 (local address 127.31.59.190:43557)
0808 13:10:56.545062 (+ 2625us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:56.545067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:56.545086 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:10:56.547409 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:56.547416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:56.547507 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:56.547623 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:56.550216 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.551080 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.554002 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.554566 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.554682 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:56.559665 (+ 4983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:56.559670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:56.559674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:56.559709 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:56.562364 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:56.563489 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:56.563495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:56.563499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:56.563640 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:56.564240 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:56.564251 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:56.564253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:56.564616 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:56.564750 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:56.565816 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0808 13:10:56.566109 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2509,"thread_start_us":2409,"threads_started":1}
I20260808 13:10:56.567376 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:56.543164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.129:46303)
0808 13:10:56.544231 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:56.544282 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:56.547056 (+ 2774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:56.547933 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:56.547947 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:56.548897 (+ 950us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:56.549996 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:56.550029 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.551310 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.551316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:56.553796 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:56.553846 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.554833 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.554842 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:56.555121 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:56.556079 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:56.556114 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:56.559424 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:56.562542 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:56.562555 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:56.562573 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:56.563335 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:56.563774 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:56.563779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:56.563784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:56.564124 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:56.564823 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:56.564830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:56.565326 (+ 496us) client_negotiation.cc:769] Sending connection context
0808 13:10:56.565678 (+ 352us) client_negotiation.cc:240] Negotiation successful
0808 13:10:56.566157 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":412,"threads_started":1}
I20260808 13:10:56.587574 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:56.573498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:46399 (local address 127.31.59.190:43557)
0808 13:10:56.573677 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:56.573682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:56.574540 (+ 858us) server_negotiation.cc:407] Connection header received
0808 13:10:56.574658 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:56.574664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:56.574754 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:56.574861 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:56.576037 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.576707 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.578682 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.579264 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.579387 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:56.581841 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:56.581848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:56.581853 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:56.581891 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:56.584694 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:56.585334 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:56.585339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:56.585341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:56.585497 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:56.585908 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:56.585914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:56.585918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:56.586308 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:56.586489 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:56.586980 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 13:10:56.587205 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:10:56.587718 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:56.573560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.129:46399)
0808 13:10:56.574440 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:56.574456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:56.574568 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:56.574913 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:56.574918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:56.575467 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:56.575890 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:56.575904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.576845 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.576850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:56.578556 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:56.578569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:56.579636 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:56.579641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:56.579767 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:56.580411 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:56.580427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:56.581596 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:56.584817 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:56.584822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:56.584827 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:56.585210 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:56.585615 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:56.585620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:56.585624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:56.585762 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:56.586429 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:56.586435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:56.586669 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 13:10:56.586838 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 13:10:56.587242 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":506,"threads_started":1}
I20260808 13:10:56.593235 7702 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:39713
I20260808 13:10:56.593264 7848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:39713 every 8 connection(s)
I20260808 13:10:56.594434 7702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:10:56.596401 7849 heartbeater.cc:347] Connected to a master server at 127.31.59.190:43557
I20260808 13:10:56.596606 7849 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:56.597234 7849 heartbeater.cc:511] Master 127.31.59.190:43557 requested a full tablet report, sending...
I20260808 13:10:56.598742 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 7702
I20260808 13:10:56.598954 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:10:56.599279 7630 ts_manager.cc:194] Registered new tserver with Master: da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713)
I20260808 13:10:56.599615 31982 external_mini_cluster.cc:1428] Setting key f8538a7c5db562bbe1851b8fabbc1950
I20260808 13:10:56.601068 7630 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:46303
WARNING: no policy specified for oryx/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
I20260808 13:10:56.653563 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:43557
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:56.991516 7853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:56.991823 7853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:56.991925 7853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:56.999334 7853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:56.999456 7853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:56.999557 7853 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:56.999665 7853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:10:57.009871 7853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:57.011195 7853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:57.012583 7853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:10:57.025188 7859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:57.025326 7860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:57.025328 7863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:57.026280 7853 server_base.cc:1034] running on GCE node
I20260808 13:10:57.026872 7853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:57.027765 7853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:57.028995 7853 hybrid_clock.cc:648] HybridClock initialized: now 1786194657028928 us; error 31 us; skew 500 ppm
I20260808 13:10:57.029306 7853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:57.032456 7853 init.cc:377] Logged in from keytab as oryx/127.31.59.130@KRBTEST.COM (short username oryx)
I20260808 13:10:57.033943 7853 webserver.cc:460] Webserver started at http://127.31.59.130:46819/ using document root <none> and password file <none>
I20260808 13:10:57.034621 7853 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:57.034724 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:57.035045 7853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:57.038067 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "c4d81b47546c17eb9f2fc4146e36b756"
server_key_iv: "a1f2c787e24f95626eb8d97684a01410"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.038869 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "c4d81b47546c17eb9f2fc4146e36b756"
server_key_iv: "a1f2c787e24f95626eb8d97684a01410"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.043920 7853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:10:57.047289 7870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:57.048538 7853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:10:57.048686 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "c4d81b47546c17eb9f2fc4146e36b756"
server_key_iv: "a1f2c787e24f95626eb8d97684a01410"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.048903 7853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:57.062685 7853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:57.066401 7853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:57.066630 7853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:57.069237 7853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:57.069339 7853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:57.069486 7853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:57.069561 7853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:57.094457 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.069930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:51253 (local address 127.31.59.190:43557)
0808 13:10:57.070153 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:57.070160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:57.071643 (+ 1483us) server_negotiation.cc:407] Connection header received
0808 13:10:57.073737 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:57.073743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:57.073836 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:57.073922 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:57.076108 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.077283 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.080326 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.080903 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.081038 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.086372 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:57.086377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:57.086380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:57.086416 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:57.090152 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.090934 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.090941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.090945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.091110 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.091735 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.091738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.091740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.092146 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:57.093594 (+ 1448us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:57.093777 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 13:10:57.094045 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 13:10:57.095073 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.070560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.130:51253)
0808 13:10:57.071472 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:57.071516 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:57.073421 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:57.074212 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:57.074224 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:57.075027 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:57.075882 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.075905 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.077547 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.077554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.080125 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.080166 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.081202 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.081211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.081558 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.082521 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:57.082561 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:57.086140 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:57.090303 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.090311 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.090327 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.090788 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.091276 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.091281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.091285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.091600 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.092325 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:57.092335 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:57.092812 (+ 477us) client_negotiation.cc:769] Sending connection context
0808 13:10:57.093175 (+ 363us) client_negotiation.cc:240] Negotiation successful
0808 13:10:57.093787 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":411,"threads_started":1}
I20260808 13:10:57.115352 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.100882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:57957 (local address 127.31.59.190:43557)
0808 13:10:57.101076 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:57.101083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:57.101671 (+ 588us) server_negotiation.cc:407] Connection header received
0808 13:10:57.101857 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:57.101860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:57.101932 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:57.102009 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:57.103408 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.104463 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.106538 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.107010 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.107120 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.109094 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:57.109101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:57.109105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:57.109157 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:57.112275 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.113161 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.113165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.113167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.113338 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.113749 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.113754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.113758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.114203 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:57.114347 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:57.114786 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 13:10:57.115002 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:10:57.115435 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.100867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.130:57957)
0808 13:10:57.101566 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:57.101589 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:57.101751 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:57.102054 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:57.102059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:57.102764 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:57.103186 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.103198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.104642 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.104652 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.106387 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.106405 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.107229 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.107235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.107448 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.108142 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:57.108159 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:57.108909 (+ 750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:57.112418 (+ 3509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.112425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.112430 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.113046 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.113478 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.113483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.113487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.113634 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.114330 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:57.114336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:57.114524 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 13:10:57.114641 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 13:10:57.115003 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":361,"threads_started":1}
I20260808 13:10:57.125749 7853 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:35649
I20260808 13:10:57.125777 7999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:35649 every 8 connection(s)
I20260808 13:10:57.127027 7853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:10:57.128949 8000 heartbeater.cc:347] Connected to a master server at 127.31.59.190:43557
I20260808 13:10:57.129133 8000 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:57.129788 8000 heartbeater.cc:511] Master 127.31.59.190:43557 requested a full tablet report, sending...
I20260808 13:10:57.131040 7630 ts_manager.cc:194] Registered new tserver with Master: 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649)
I20260808 13:10:57.131848 7630 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:51253
I20260808 13:10:57.132349 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 7853
I20260808 13:10:57.132566 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:10:57.132980 31982 external_mini_cluster.cc:1428] Setting key eef2316d7e463dc1b505ee3e441c9d7c
WARNING: no policy specified for oryx/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
I20260808 13:10:57.186914 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:43557
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:10:57.572340 8004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:10:57.572639 8004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:10:57.572773 8004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:10:57.579929 8004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:10:57.580061 8004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:10:57.580201 8004 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:10:57.580314 8004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:10:57.590250 8004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.190:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:10:57.591674 8004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:10:57.593081 8004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:10:57.603912 7849 heartbeater.cc:503] Master 127.31.59.190:43557 was elected leader, sending a full tablet report...
W20260808 13:10:57.606343 8010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:57.606534 8009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:10:57.606828 8012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:10:57.607223 8004 server_base.cc:1034] running on GCE node
I20260808 13:10:57.607836 8004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:10:57.608733 8004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:10:57.609951 8004 hybrid_clock.cc:648] HybridClock initialized: now 1786194657609878 us; error 44 us; skew 500 ppm
I20260808 13:10:57.610255 8004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:10:57.613555 8004 init.cc:377] Logged in from keytab as oryx/127.31.59.131@KRBTEST.COM (short username oryx)
I20260808 13:10:57.615259 8004 webserver.cc:460] Webserver started at http://127.31.59.131:45533/ using document root <none> and password file <none>
I20260808 13:10:57.615957 8004 fs_manager.cc:359] Metadata directory not provided
I20260808 13:10:57.616058 8004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:10:57.616376 8004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:10:57.619305 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "216a17a61a6f8917b8c9f0e73fd49832"
server_key_iv: "71fb8bc77fc19d7e5e03eaed816d9b25"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.620093 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "216a17a61a6f8917b8c9f0e73fd49832"
server_key_iv: "71fb8bc77fc19d7e5e03eaed816d9b25"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.625193 8004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:10:57.628544 8020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:57.629825 8004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:10:57.629982 8004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
format_stamp: "Formatted at 2026-08-08 13:10:57 on dist-test-slave-49h3"
server_key: "216a17a61a6f8917b8c9f0e73fd49832"
server_key_iv: "71fb8bc77fc19d7e5e03eaed816d9b25"
server_key_version: "encryptionkey@0"
I20260808 13:10:57.630183 8004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:10:57.644307 8004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:10:57.648869 8004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:10:57.649154 8004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:10:57.652834 8004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:10:57.652943 8004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:10:57.653087 8004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:10:57.653144 8004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:57.688772 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.653686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.131:39763)
0808 13:10:57.663549 (+ 9863us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:57.663612 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:57.667359 (+ 3747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:57.668278 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:57.668287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:57.668867 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:57.669707 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.669738 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.671325 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.671332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.673955 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.673978 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.675167 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.675177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.675465 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.676321 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:57.676344 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:57.679547 (+ 3203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:57.682702 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.682716 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.682737 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.683391 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.683888 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.683894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.683901 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.684226 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.684924 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:57.684933 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:57.685558 (+ 625us) client_negotiation.cc:769] Sending connection context
0808 13:10:57.685957 (+ 399us) client_negotiation.cc:240] Negotiation successful
0808 13:10:57.686702 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9477,"thread_start_us":11655,"threads_started":1}
I20260808 13:10:57.689512 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.653252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39763 (local address 127.31.59.190:43557)
0808 13:10:57.665918 (+ 12666us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:57.665925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:57.665958 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:10:57.667759 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:57.667766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:57.667875 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:57.667995 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:57.669976 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.671124 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.674147 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.674677 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.674816 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.679792 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:57.679797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:57.679801 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:57.679837 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:57.682564 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.683585 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.683590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.683592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.683743 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.684397 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.684402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.684405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.684750 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:57.684916 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:57.688800 (+ 3884us) server_negotiation.cc:299] Negotiation successful
0808 13:10:57.689084 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12499,"thread_start_us":12368,"threads_started":1}
I20260808 13:10:57.705408 8004 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:39337
I20260808 13:10:57.705430 8147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:39337 every 8 connection(s)
I20260808 13:10:57.706931 8004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:10:57.709323 8148 heartbeater.cc:347] Connected to a master server at 127.31.59.190:43557
I20260808 13:10:57.709617 8148 heartbeater.cc:464] Registering TS with master...
I20260808 13:10:57.710774 8148 heartbeater.cc:511] Master 127.31.59.190:43557 requested a full tablet report, sending...
I20260808 13:10:57.712270 7630 ts_manager.cc:194] Registered new tserver with Master: 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:57.712374 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.696456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:45535 (local address 127.31.59.190:43557)
0808 13:10:57.696700 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:57.696706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:57.697406 (+ 700us) server_negotiation.cc:407] Connection header received
0808 13:10:57.697586 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:57.697592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:57.697717 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:57.697825 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:57.699523 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.700270 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.702603 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.703256 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.703363 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.706149 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:57.706154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:57.706157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:57.706190 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:57.709113 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.710258 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.710263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.710267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.710432 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:57.710815 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:57.710820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:57.710824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:57.711189 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:57.711335 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:57.711760 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 13:10:57.711983 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 13:10:57.712713 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:57.696428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.31.59.131:45535)
0808 13:10:57.697270 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:57.697289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:57.697423 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:57.698014 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:57.698020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:57.698754 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:57.699351 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.699369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.700425 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.700432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.702385 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:57.702403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.703673 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.703679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:57.703831 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:57.704576 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:57.704601 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:57.705955 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:57.709265 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.709271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.709276 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.710119 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.710535 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:57.710548 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:57.710550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:57.710708 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:57.711325 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:57.711328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:57.711495 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 13:10:57.711619 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 13:10:57.711932 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":515,"threads_started":1}
I20260808 13:10:57.713088 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8004
I20260808 13:10:57.713294 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:10:57.713311 7630 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:39763
I20260808 13:10:57.713809 31982 external_mini_cluster.cc:1428] Setting key 0b403d8c3045a33d92e3dacd15feb218
I20260808 13:10:57.717422 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:10:57 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:10:58.134349 8000 heartbeater.cc:503] Master 127.31.59.190:43557 was elected leader, sending a full tablet report...
I20260808 13:10:58.717284 8148 heartbeater.cc:503] Master 127.31.59.190:43557 was elected leader, sending a full tablet report...
W20260808 13:10:58.761324 8156 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.992s user 0.004s sys 0.015s
W20260808 13:10:58.762552 8029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:57.763834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.31.59.190:43557)
0808 13:10:57.764068 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:57.764075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:57.764270 (+ 195us) server_negotiation.cc:407] Connection header received
0808 13:10:57.764462 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:57.764466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:57.764531 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:57.764616 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:57.766062 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.766809 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.768050 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:57.768466 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:57.768576 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.762319 (+993743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54594: BlockingRecv error: recv got EOF from 127.0.0.1:54594 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 13:10:58 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:58.783753 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.770601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.31.59.190:43557)
0808 13:10:58.770835 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:58.770841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:58.771055 (+ 214us) server_negotiation.cc:407] Connection header received
0808 13:10:58.771239 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:58.771245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:58.771318 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:58.771403 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:58.772624 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.773353 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.774640 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.775119 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.775233 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.778892 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:58.778898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:58.778901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:58.778935 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:58.781253 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:58.781906 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:58.781912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:58.781916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:58.782045 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:58.782408 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:58.782411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:58.782413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:58.782676 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:58.782969 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:58.783217 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 13:10:58.783454 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:10:58.787849 7630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:58.794915 7630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:58.818542 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.807188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39337 (local address 127.0.0.1:36496)
0808 13:10:58.809647 (+ 2459us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:58.809700 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:58.810390 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:58.811334 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:58.811339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:58.811385 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:58.811735 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.811757 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.813428 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.813432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.815657 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.815672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.817512 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.817518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.817702 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.817768 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 13:10:58.817914 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 13:10:58.818095 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1720,"mutex_wait_us":58,"thread_start_us":2039,"threads_started":1}
I20260808 13:10:58.819414 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.807846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.31.59.130:35649)
0808 13:10:58.809948 (+ 2102us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:58.809957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:58.809985 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:10:58.810932 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:58.810942 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:58.811188 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:58.811353 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:58.812304 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.813546 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.816245 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.817690 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.817824 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.818025 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:58.818964 (+ 939us) server_negotiation.cc:299] Negotiation successful
0808 13:10:58.819135 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1947,"thread_start_us":1808,"threads_started":1}
I20260808 13:10:58.819453 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.809774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:39713 (local address 127.0.0.1:46106)
0808 13:10:58.810436 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:58.810452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:58.810543 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:58.812784 (+ 2241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:58.812789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:58.812816 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:58.813134 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.813156 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.814911 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.814917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.816752 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.816764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.818671 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.818677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.818833 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.818862 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 13:10:58.818943 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 13:10:58.819059 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":3,"thread_start_us":384,"threads_started":1}
I20260808 13:10:58.819622 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.807378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.31.59.131:39337)
0808 13:10:58.808582 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:58.808592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:58.809808 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 13:10:58.810626 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:58.810634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:58.810888 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:58.811064 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:58.811903 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.813319 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.815805 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.817290 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.817430 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.817680 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:58.819248 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0808 13:10:58.819370 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":895,"threads_started":1}
I20260808 13:10:58.820161 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.810264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.31.59.129:39713)
0808 13:10:58.812286 (+ 2022us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:58.812294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:58.812319 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:10:58.812404 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:58.812411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:58.812599 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:58.812782 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:58.813320 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.814723 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.816899 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.818475 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.818607 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.818795 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:58.819712 (+ 917us) server_negotiation.cc:299] Negotiation successful
0808 13:10:58.819877 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1868,"thread_start_us":1719,"threads_started":1}
I20260808 13:10:58.821805 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:58.807330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:35649 (local address 127.0.0.1:51636)
0808 13:10:58.809892 (+ 2562us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:58.809906 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:58.810078 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:58.811626 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:58.811631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:58.811657 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:58.812088 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.812098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.813701 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.813706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.816034 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:58.816048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:58.817908 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:58.817915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:58.818297 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:58.818329 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:10:58.818423 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 13:10:58.819768 (+ 1345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2024,"thread_start_us":1632,"threads_started":1}
I20260808 13:10:58.823616 8073 tablet_service.cc:1467] Processing CreateTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (DEFAULT_TABLE table=test-table [id=268fa3e97694456980803f38cbbb2cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:58.823737 7773 tablet_service.cc:1467] Processing CreateTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (DEFAULT_TABLE table=test-table [id=268fa3e97694456980803f38cbbb2cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:58.824927 8073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:58.825086 7773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:58.825407 7923 tablet_service.cc:1467] Processing CreateTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (DEFAULT_TABLE table=test-table [id=268fa3e97694456980803f38cbbb2cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:58.826713 7923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:58.836987 8175 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Bootstrap starting.
I20260808 13:10:58.839020 8176 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Bootstrap starting.
I20260808 13:10:58.840373 8175 tablet_bootstrap.cc:654] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:58.840691 8177 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Bootstrap starting.
I20260808 13:10:58.841818 8175 log.cc:824] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:58.842468 8176 tablet_bootstrap.cc:654] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:58.843689 8176 log.cc:824] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:58.843860 8177 tablet_bootstrap.cc:654] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:58.845024 8175 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: No bootstrap required, opened a new log
I20260808 13:10:58.845139 8177 log.cc:824] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Log is configured to *not* fsync() on all Append() calls
I20260808 13:10:58.845309 8175 ts_tablet_manager.cc:1397] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 13:10:58.846596 8176 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: No bootstrap required, opened a new log
I20260808 13:10:58.846837 8176 ts_tablet_manager.cc:1397] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260808 13:10:58.848016 8177 tablet_bootstrap.cc:492] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: No bootstrap required, opened a new log
I20260808 13:10:58.848271 8177 ts_tablet_manager.cc:1397] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:10:58.854461 8176 raft_consensus.cc:348] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.854830 8176 raft_consensus.cc:374] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:58.854954 8176 raft_consensus.cc:729] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Initialized, Role: FOLLOWER
I20260808 13:10:58.855556 8176 consensus_queue.cc:260] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.855682 8175 raft_consensus.cc:348] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.856138 8175 raft_consensus.cc:374] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:58.856272 8175 raft_consensus.cc:729] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7f94c02f0d42abb1fc457cb7e40436, State: Initialized, Role: FOLLOWER
I20260808 13:10:58.856859 8175 consensus_queue.cc:260] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.858655 8175 ts_tablet_manager.cc:1428] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260808 13:10:58.858425 8177 raft_consensus.cc:348] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.858860 8177 raft_consensus.cc:374] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:58.858976 8177 raft_consensus.cc:729] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Initialized, Role: FOLLOWER
I20260808 13:10:58.859812 8176 ts_tablet_manager.cc:1428] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Time spent starting tablet: real 0.013s user 0.003s sys 0.008s
I20260808 13:10:58.859692 8177 consensus_queue.cc:260] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:58.861491 8177 ts_tablet_manager.cc:1428] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
W20260808 13:10:58.879740 8001 tablet.cc:2369] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:10:58.961268 8149 tablet.cc:2369] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:59.090458 8181 raft_consensus.cc:482] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:59.090821 8181 raft_consensus.cc:504] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:59.092700 8181 leader_election.cc:288] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337), 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649)
W20260808 13:10:59.098762 7850 tablet.cc:2369] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:10:59.100895 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.093137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:38051 (local address 127.31.59.131:39337)
0808 13:10:59.093379 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.093386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.094225 (+ 839us) server_negotiation.cc:407] Connection header received
0808 13:10:59.094410 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.094414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.094495 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.094594 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:59.095622 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.096604 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.099160 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.100008 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.100175 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.100254 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.100528 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.100655 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:10:59.101079 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.093156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39337 (local address 127.31.59.129:38051)
0808 13:10:59.094066 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.094086 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.094276 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.094624 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.094629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.094677 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:59.095479 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.095489 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.096725 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.096730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.098920 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.098937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.100224 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.100230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.100379 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.100403 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.100485 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.100623 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":464,"spinlock_wait_cycles":58112,"thread_start_us":478,"threads_started":1}
I20260808 13:10:59.101119 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.093195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:36201 (local address 127.31.59.130:35649)
0808 13:10:59.093389 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.093394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.094220 (+ 826us) server_negotiation.cc:407] Connection header received
0808 13:10:59.094408 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.094414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.094495 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.094611 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:59.095315 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.096310 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.099084 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.100111 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.100260 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.100335 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.100628 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.100762 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:10:59.101126 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.093232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:35649 (local address 127.31.59.129:36201)
0808 13:10:59.094124 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.094138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.094267 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.094624 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.094629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.094675 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:59.095154 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.095172 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.096440 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.096444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.098931 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.098940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.100341 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.100348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.100478 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.100501 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.100583 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.100702 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":75,"spinlock_wait_cycles":12416,"thread_start_us":398,"threads_started":1}
I20260808 13:10:59.102171 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c17b20eb7ba4247a88f0a7a5cd49f72" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:10:59.102257 7945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c17b20eb7ba4247a88f0a7a5cd49f72" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" is_pre_election: true
I20260808 13:10:59.102661 8094 raft_consensus.cc:2454] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.102773 7945 raft_consensus.cc:2454] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.103250 7721 leader_election.cc:302] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7f94c02f0d42abb1fc457cb7e40436, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.103608 8181 raft_consensus.cc:2781] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:59.103763 8181 raft_consensus.cc:482] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:59.103861 8181 raft_consensus.cc:3037] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.106163 8181 raft_consensus.cc:504] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:59.106804 8181 leader_election.cc:288] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Requested vote from peers 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337), 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649)
I20260808 13:10:59.107223 7945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c17b20eb7ba4247a88f0a7a5cd49f72" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
I20260808 13:10:59.107257 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c17b20eb7ba4247a88f0a7a5cd49f72" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
I20260808 13:10:59.107421 7945 raft_consensus.cc:3037] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.107463 8094 raft_consensus.cc:3037] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.109831 7945 raft_consensus.cc:2454] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:10:59.109835 8094 raft_consensus.cc:2454] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:10:59.110203 7721 leader_election.cc:302] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7f94c02f0d42abb1fc457cb7e40436, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.110504 8181 raft_consensus.cc:2781] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:59.111178 8181 raft_consensus.cc:686] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Becoming Leader. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Running, Role: LEADER
I20260808 13:10:59.111726 8181 consensus_queue.cc:237] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } }
I20260808 13:10:59.116868 7629 catalog_manager.cc:5495] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 reported cstate change: term changed from 0 to 1, leader changed from <none> to da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } health_report { overall_health: HEALTHY } } }
I20260808 13:10:59.190475 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.185584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.31.59.129:39713)
0808 13:10:59.185771 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.185775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.185927 (+ 152us) server_negotiation.cc:407] Connection header received
0808 13:10:59.186117 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.186120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.186202 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.186327 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:59.186603 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.187382 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.188649 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.189049 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.189142 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.189498 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.190054 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.190162 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.190204 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.190270 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:10:59.198840 8094 raft_consensus.cc:1261] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.198841 7945 raft_consensus.cc:1261] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.199779 8188 consensus_queue.cc:1035] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.200121 8181 consensus_queue.cc:1035] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.214128 8193 mvcc.cc:205] Tried to move back new op lower bound from 7316253324068237312 to 7316253323726303232. Current Snapshot: MvccSnapshot[applied={T|T < 7316253324068237312}]
I20260808 13:10:59.214790 8192 mvcc.cc:205] Tried to move back new op lower bound from 7316253324068237312 to 7316253323726303232. Current Snapshot: MvccSnapshot[applied={T|T < 7316253324068237312}]
I20260808 13:10:59.225288 7627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:59.225629 7627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:59.229288 7627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Sending DeleteTablet for 3 replicas of tablet 7c17b20eb7ba4247a88f0a7a5cd49f72
I20260808 13:10:59.230408 8073 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36496
I20260808 13:10:59.230492 7923 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260808 13:10:59.231381 7773 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 13:10:59.231549 7629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:59.232280 7629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:59.241634 8072 tablet_service.cc:1467] Processing CreateTablet for tablet acc87c7778a5489498fb4c860874ee8a (DEFAULT_TABLE table=test-table [id=5f1be3e2b559439bbacb9f524bab1d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.241778 7922 tablet_service.cc:1467] Processing CreateTablet for tablet acc87c7778a5489498fb4c860874ee8a (DEFAULT_TABLE table=test-table [id=5f1be3e2b559439bbacb9f524bab1d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.242290 8072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc87c7778a5489498fb4c860874ee8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.242413 7922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc87c7778a5489498fb4c860874ee8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.246326 7772 tablet_service.cc:1467] Processing CreateTablet for tablet acc87c7778a5489498fb4c860874ee8a (DEFAULT_TABLE table=test-table [id=5f1be3e2b559439bbacb9f524bab1d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.246995 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc87c7778a5489498fb4c860874ee8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.254940 8175 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Bootstrap starting.
I20260808 13:10:59.256595 8175 tablet_bootstrap.cc:654] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.265161 8177 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Bootstrap starting.
I20260808 13:10:59.267107 8177 tablet_bootstrap.cc:654] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.272789 8176 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Bootstrap starting.
I20260808 13:10:59.274842 8176 tablet_bootstrap.cc:654] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.276796 8175 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: No bootstrap required, opened a new log
I20260808 13:10:59.276960 8175 ts_tablet_manager.cc:1397] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.009s
I20260808 13:10:59.279443 8202 tablet_replica.cc:330] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: stopping tablet replica
I20260808 13:10:59.279943 8202 raft_consensus.cc:2229] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:59.280663 8202 raft_consensus.cc:2258] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.281522 8175 raft_consensus.cc:348] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.281783 8175 raft_consensus.cc:374] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.281883 8175 raft_consensus.cc:729] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7f94c02f0d42abb1fc457cb7e40436, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.282163 8175 consensus_queue.cc:260] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.283453 8175 ts_tablet_manager.cc:1428] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260808 13:10:59.284399 8176 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: No bootstrap required, opened a new log
I20260808 13:10:59.284561 8176 ts_tablet_manager.cc:1397] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260808 13:10:59.285419 8176 raft_consensus.cc:348] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.285693 8176 raft_consensus.cc:374] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.285796 8176 raft_consensus.cc:729] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.286120 8176 consensus_queue.cc:260] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.286736 8176 ts_tablet_manager.cc:1428] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 13:10:59.287427 8200 tablet_replica.cc:330] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: stopping tablet replica
I20260808 13:10:59.287865 8200 raft_consensus.cc:2229] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:59.288358 8200 raft_consensus.cc:2258] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.289367 8202 ts_tablet_manager.cc:1905] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.290252 8200 ts_tablet_manager.cc:1905] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.294656 8177 tablet_bootstrap.cc:492] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: No bootstrap required, opened a new log
I20260808 13:10:59.294836 8177 ts_tablet_manager.cc:1397] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.006s
I20260808 13:10:59.296228 8177 raft_consensus.cc:348] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.296499 8177 raft_consensus.cc:374] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.296592 8177 raft_consensus.cc:729] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.296913 8177 consensus_queue.cc:260] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.297411 8200 ts_tablet_manager.cc:1918] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:59.297606 8200 log.cc:1196] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/7c17b20eb7ba4247a88f0a7a5cd49f72
I20260808 13:10:59.298131 8200 ts_tablet_manager.cc:1939] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting consensus metadata
I20260808 13:10:59.299963 7615 catalog_manager.cc:4841] TS 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337): tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (table test-table [id=268fa3e97694456980803f38cbbb2cea]) successfully deleted
I20260808 13:10:59.301011 8177 ts_tablet_manager.cc:1428] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260808 13:10:59.303295 8201 tablet_replica.cc:330] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: stopping tablet replica
I20260808 13:10:59.303725 8201 raft_consensus.cc:2229] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:59.303848 8202 ts_tablet_manager.cc:1918] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:59.304023 8202 log.cc:1196] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/7c17b20eb7ba4247a88f0a7a5cd49f72
I20260808 13:10:59.304220 8201 raft_consensus.cc:2258] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.304579 8202 ts_tablet_manager.cc:1939] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting consensus metadata
I20260808 13:10:59.306262 8201 ts_tablet_manager.cc:1905] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.306358 7616 catalog_manager.cc:4841] TS da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713): tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (table test-table [id=268fa3e97694456980803f38cbbb2cea]) successfully deleted
I20260808 13:10:59.310986 8201 ts_tablet_manager.cc:1918] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:10:59.311146 8201 log.cc:1196] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/7c17b20eb7ba4247a88f0a7a5cd49f72
I20260808 13:10:59.311619 8201 ts_tablet_manager.cc:1939] T 7c17b20eb7ba4247a88f0a7a5cd49f72 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting consensus metadata
I20260808 13:10:59.312888 7616 catalog_manager.cc:4841] TS 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649): tablet 7c17b20eb7ba4247a88f0a7a5cd49f72 (table test-table [id=268fa3e97694456980803f38cbbb2cea]) successfully deleted
I20260808 13:10:59.377622 8188 raft_consensus.cc:482] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:59.377903 8188 raft_consensus.cc:504] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.378463 8184 raft_consensus.cc:482] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:59.378629 8188 leader_election.cc:288] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.378657 8184 raft_consensus.cc:504] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.379390 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:10:59.379393 7945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" is_pre_election: true
I20260808 13:10:59.379642 8094 raft_consensus.cc:2454] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.379638 7945 raft_consensus.cc:2454] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.380105 7721 leader_election.cc:302] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7f94c02f0d42abb1fc457cb7e40436, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.380380 8188 raft_consensus.cc:2781] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:59.380481 8188 raft_consensus.cc:482] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:59.380555 8184 leader_election.cc:288] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.380595 8188 raft_consensus.cc:3037] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.384256 8188 raft_consensus.cc:504] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.385438 7945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
I20260808 13:10:59.385679 7945 raft_consensus.cc:3037] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.389101 7945 raft_consensus.cc:2454] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:10:59.389626 7722 leader_election.cc:302] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.390009 8181 raft_consensus.cc:2781] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:59.391981 8181 raft_consensus.cc:686] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Becoming Leader. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Running, Role: LEADER
I20260808 13:10:59.392335 8181 consensus_queue.cc:237] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.394320 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
I20260808 13:10:59.394568 8094 raft_consensus.cc:3037] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.397890 8094 raft_consensus.cc:2454] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:10:59.398576 8188 leader_election.cc:288] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Requested vote from peers 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.400785 7627 catalog_manager.cc:5495] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 reported cstate change: term changed from 0 to 1, leader changed from <none> to da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:59.402390 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.381659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:40627 (local address 127.31.59.129:39713)
0808 13:10:59.381819 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.381825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.393214 (+ 11389us) server_negotiation.cc:407] Connection header received
0808 13:10:59.394966 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.394972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.395062 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.395125 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:59.395965 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.397032 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.399746 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.401035 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.401169 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.401261 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.401905 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.402059 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 13:10:59.403120 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.380865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:39713 (local address 127.31.59.130:40627)
0808 13:10:59.393101 (+ 12236us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.393123 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.394786 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.395350 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.395355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.395391 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:59.395772 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.395786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.397211 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.397216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.399540 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.399557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.401491 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.401499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.401704 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.401742 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.402624 (+ 882us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.402779 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12092,"threads_started":1}
I20260808 13:10:59.405385 7794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" is_pre_election: true
I20260808 13:10:59.409376 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.381537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:42081 (local address 127.31.59.131:39337)
0808 13:10:59.381717 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.381723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.403444 (+ 21721us) server_negotiation.cc:407] Connection header received
0808 13:10:59.403771 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.403778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.403853 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.403913 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:10:59.404444 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.405576 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.407780 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.408545 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.408675 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.408727 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.409056 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.409149 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 13:10:59.409507 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.381136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39337 (local address 127.31.59.130:42081)
0808 13:10:59.403347 (+ 22211us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.403368 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.403546 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.404099 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.404104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.404130 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:10:59.404283 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.404294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.405770 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.405775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.407612 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.407628 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.408710 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.408717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.408900 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.408935 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.409026 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.409156 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22094,"thread_start_us":12090,"threads_started":1}
I20260808 13:10:59.410064 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc87c7778a5489498fb4c860874ee8a" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:10:59.410364 8094 raft_consensus.cc:2379] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ab0e53f4a4a4363b77229b52e3c0877 in current term 1: Already voted for candidate da81fc3d2bff40f68b2ed1adeea6d329 in this term.
I20260808 13:10:59.410851 7873 leader_election.cc:302] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877; no voters: 1c7f94c02f0d42abb1fc457cb7e40436, da81fc3d2bff40f68b2ed1adeea6d329
I20260808 13:10:59.411310 8184 raft_consensus.cc:2726] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 08 13:10:59 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194655, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.190@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:59.453676 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.439776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.31.59.190:43557)
0808 13:10:59.440712 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.440717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.440737 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:59.440789 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.440793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.440843 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.440926 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:59.442072 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.442848 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.444047 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.444468 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.444571 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.448130 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:59.448135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:59.448138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:59.448168 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:59.450718 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:59.451576 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:59.451581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:59.451585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:59.451702 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:59.452093 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:59.452096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:59.452098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:59.452368 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:59.452473 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.453012 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.453348 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":799,"thread_start_us":640,"threads_started":1}
I20260808 13:10:59.453752 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.439722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:43557 (local address 127.0.0.1:54606)
0808 13:10:59.440422 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.440442 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.440605 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.440971 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.440979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.441723 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:10:59.441928 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.441939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.442966 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.442970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.443910 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.443921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.444620 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.444625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.444800 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.445472 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:10:59.445503 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:10:59.447933 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:10:59.450866 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:59.450880 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:59.450887 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:59.451456 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:59.451801 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:10:59.451806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:10:59.451809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:10:59.451984 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:10:59.452494 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:10:59.452502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:10:59.452756 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.452936 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.453238 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":348,"threads_started":1}
I20260808 13:10:59.459160 7627 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:54606:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:10:59.460114 7627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:59.466974 7772 tablet_service.cc:1467] Processing CreateTablet for tablet ac51f8a81a2946a88fe7f38eedc48ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d223ea38e0e84b8b991cff576bd1d5e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:59.467177 8073 tablet_service.cc:1467] Processing CreateTablet for tablet ac51f8a81a2946a88fe7f38eedc48ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d223ea38e0e84b8b991cff576bd1d5e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:59.467507 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac51f8a81a2946a88fe7f38eedc48ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.467774 8073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac51f8a81a2946a88fe7f38eedc48ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.473037 8176 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: Bootstrap starting.
I20260808 13:10:59.473637 7923 tablet_service.cc:1467] Processing CreateTablet for tablet ac51f8a81a2946a88fe7f38eedc48ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d223ea38e0e84b8b991cff576bd1d5e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:10:59.474208 7923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac51f8a81a2946a88fe7f38eedc48ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.474779 8175 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: Bootstrap starting.
I20260808 13:10:59.474900 8176 tablet_bootstrap.cc:654] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.476379 8175 tablet_bootstrap.cc:654] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.483016 8177 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: Bootstrap starting.
I20260808 13:10:59.484721 8177 tablet_bootstrap.cc:654] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.485332 8176 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: No bootstrap required, opened a new log
I20260808 13:10:59.485597 8176 ts_tablet_manager.cc:1397] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260808 13:10:59.486518 8176 raft_consensus.cc:348] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.486996 8176 raft_consensus.cc:374] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.487129 8176 raft_consensus.cc:729] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.487489 8176 consensus_queue.cc:260] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.488051 8181 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.488440 8181 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.489913 8176 ts_tablet_manager.cc:1428] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 13:10:59.494971 8175 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: No bootstrap required, opened a new log
I20260808 13:10:59.495090 8177 tablet_bootstrap.cc:492] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: No bootstrap required, opened a new log
I20260808 13:10:59.495143 8175 ts_tablet_manager.cc:1397] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.009s
I20260808 13:10:59.495277 8177 ts_tablet_manager.cc:1397] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260808 13:10:59.495844 8175 raft_consensus.cc:348] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.496088 8175 raft_consensus.cc:374] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.496167 8175 raft_consensus.cc:729] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7f94c02f0d42abb1fc457cb7e40436, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.496480 8177 raft_consensus.cc:348] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.496438 8175 consensus_queue.cc:260] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.496721 8177 raft_consensus.cc:374] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.496793 8177 raft_consensus.cc:729] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.497017 8177 consensus_queue.cc:260] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.496970 8183 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.497355 8183 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.497632 8177 ts_tablet_manager.cc:1428] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 13:10:59.497721 8184 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.498008 8184 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.498497 8175 ts_tablet_manager.cc:1428] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 13:10:59.556028 8184 raft_consensus.cc:482] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:59.556245 8184 raft_consensus.cc:504] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.556916 8184 leader_election.cc:288] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.557612 7794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac51f8a81a2946a88fe7f38eedc48ddc" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" is_pre_election: true
I20260808 13:10:59.557667 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac51f8a81a2946a88fe7f38eedc48ddc" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:10:59.557966 8094 raft_consensus.cc:2454] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 0.
I20260808 13:10:59.557971 7794 raft_consensus.cc:2454] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 0.
I20260808 13:10:59.558434 7874 leader_election.cc:302] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.558717 8184 raft_consensus.cc:2781] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:59.558837 8184 raft_consensus.cc:482] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:59.558926 8184 raft_consensus.cc:3037] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.561051 8184 raft_consensus.cc:504] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.561628 8184 leader_election.cc:288] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 election: Requested vote from peers da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.562013 7794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac51f8a81a2946a88fe7f38eedc48ddc" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da81fc3d2bff40f68b2ed1adeea6d329"
I20260808 13:10:59.562266 7794 raft_consensus.cc:3037] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.562191 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac51f8a81a2946a88fe7f38eedc48ddc" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
I20260808 13:10:59.562431 8094 raft_consensus.cc:3037] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.565208 8094 raft_consensus.cc:2454] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 1.
I20260808 13:10:59.565636 7794 raft_consensus.cc:2454] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 1.
I20260808 13:10:59.565686 7873 leader_election.cc:302] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7f94c02f0d42abb1fc457cb7e40436, 6ab0e53f4a4a4363b77229b52e3c0877; no voters:
I20260808 13:10:59.565973 8184 raft_consensus.cc:2781] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:10:59.566533 8184 raft_consensus.cc:686] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 LEADER]: Becoming Leader. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Running, Role: LEADER
I20260808 13:10:59.567040 8184 consensus_queue.cc:237] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.569314 8220 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: TxnStatusTablet state changed. Reason: New leader 6ab0e53f4a4a4363b77229b52e3c0877. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.569900 8220 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:59.570990 8222 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:10:59.571177 8222 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:10:59.571358 7627 catalog_manager.cc:5495] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260808 13:10:59.593668 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.586986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:35649 (local address 127.0.0.1:51648)
0808 13:10:59.587216 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.587234 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.587353 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.587947 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.587951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.587964 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:59.588147 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.588155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.589510 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.589515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.590872 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.590885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.591875 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.591881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.592069 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.592142 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.593107 (+ 965us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.593127 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.593189 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.593334 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:10:59.594448 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.586916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51648 (local address 127.31.59.130:35649)
0808 13:10:59.587347 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.587354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.587374 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:10:59.587572 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.587578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.587660 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.587772 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:59.588285 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.589328 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.591027 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.591614 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.591706 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.592338 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.592950 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.593058 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.594020 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.594114 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
I20260808 13:10:59.599505 8094 raft_consensus.cc:1261] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Refusing update from remote peer 6ab0e53f4a4a4363b77229b52e3c0877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.599774 7794 raft_consensus.cc:1261] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Refusing update from remote peer 6ab0e53f4a4a4363b77229b52e3c0877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.600131 8184 consensus_queue.cc:1035] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.600696 8220 consensus_queue.cc:1035] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.612923 8183 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: TxnStatusTablet state changed. Reason: New leader 6ab0e53f4a4a4363b77229b52e3c0877. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.613215 8183 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.616536 8220 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.616806 8220 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:59.628784 8183 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.629025 8183 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 1c7f94c02f0d42abb1fc457cb7e40436: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.632578 8181 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: TxnStatusTablet state changed. Reason: New leader 6ab0e53f4a4a4363b77229b52e3c0877. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.632918 8181 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.636483 8184 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.636812 8184 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P 6ab0e53f4a4a4363b77229b52e3c0877: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:10:59.639173 8191 mvcc.cc:205] Tried to move back new op lower bound from 7316253325709930496 to 7316253325591498752. Current Snapshot: MvccSnapshot[applied={T|T < 7316253325709930496}]
I20260808 13:10:59.642284 8181 tablet_replica.cc:439] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } } }
I20260808 13:10:59.642558 8181 tablet_replica.cc:442] T ac51f8a81a2946a88fe7f38eedc48ddc P da81fc3d2bff40f68b2ed1adeea6d329: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:10:59.655738 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.648606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:38867 (local address 127.31.59.130:35649)
0808 13:10:59.648819 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.648826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.649018 (+ 192us) server_negotiation.cc:407] Connection header received
0808 13:10:59.649340 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.649346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.649473 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.649572 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:59.650239 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.651392 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.653598 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.654258 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.654386 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.654939 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.655082 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.655195 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.655341 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.655423 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:10:59.655895 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.648134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:35649 (local address 127.31.59.129:38867)
0808 13:10:59.648908 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.648932 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.649163 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.649603 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.649608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.649621 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:59.650046 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.650059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.651539 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.651546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.653375 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.653395 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.654461 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.654468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.654650 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.654737 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.655202 (+ 465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.655214 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.655306 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.655456 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":691,"threads_started":1}
I20260808 13:10:59.665774 8094 raft_consensus.cc:1261] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.666316 7945 raft_consensus.cc:1261] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:10:59.666651 8181 consensus_queue.cc:1035] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.667160 8181 consensus_queue.cc:1035] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:10:59.745936 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.735536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:44997 (local address 127.31.59.129:39713)
0808 13:10:59.735718 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.735723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.735847 (+ 124us) server_negotiation.cc:407] Connection header received
0808 13:10:59.736847 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.736854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.736939 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.737000 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:10:59.738473 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.739673 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.743586 (+ 3913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.744287 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.744384 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.745069 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.745201 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.745261 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.745528 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.745621 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:10:59.746994 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.735006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:39713 (local address 127.31.59.130:44997)
0808 13:10:59.735741 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 13:10:59.735767 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:10:59.736499 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:10:59.737225 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:10:59.737230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:10:59.737243 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:10:59.737986 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.738003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.739891 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.739897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.743404 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:10:59.743421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.744582 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.744588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:10:59.744783 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.744860 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:10:59.745349 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:10:59.745359 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:10:59.746408 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0808 13:10:59.746577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":711,"threads_started":1}
I20260808 13:10:59.778684 7629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:59.778975 7629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:10:59.781970 7629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Sending DeleteTablet for 3 replicas of tablet acc87c7778a5489498fb4c860874ee8a
I20260808 13:10:59.782810 8073 tablet_service.cc:1514] Processing DeleteTablet for tablet acc87c7778a5489498fb4c860874ee8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36496
I20260808 13:10:59.782832 7772 tablet_service.cc:1514] Processing DeleteTablet for tablet acc87c7778a5489498fb4c860874ee8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 13:10:59.782968 7923 tablet_service.cc:1514] Processing DeleteTablet for tablet acc87c7778a5489498fb4c860874ee8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:10:59 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260808 13:10:59.783087 8200 tablet_replica.cc:330] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: stopping tablet replica
I20260808 13:10:59.783257 8200 raft_consensus.cc:2229] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:59.783265 8201 tablet_replica.cc:330] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: stopping tablet replica
I20260808 13:10:59.783394 8200 raft_consensus.cc:2258] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.783463 8201 raft_consensus.cc:2229] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:10:59.783668 8201 raft_consensus.cc:2258] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.784951 8201 ts_tablet_manager.cc:1905] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.785589 8202 tablet_replica.cc:330] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: stopping tablet replica
I20260808 13:10:59.785794 8202 raft_consensus.cc:2229] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:10:59.786119 8202 raft_consensus.cc:2258] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:10:59.786268 8200 ts_tablet_manager.cc:1905] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.787210 8202 ts_tablet_manager.cc:1905] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:10:59.791776 8200 ts_tablet_manager.cc:1918] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:59.791878 8201 ts_tablet_manager.cc:1918] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:59.791985 8200 log.cc:1196] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/acc87c7778a5489498fb4c860874ee8a
I20260808 13:10:59.792040 8201 log.cc:1196] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/acc87c7778a5489498fb4c860874ee8a
I20260808 13:10:59.792404 8200 ts_tablet_manager.cc:1939] T acc87c7778a5489498fb4c860874ee8a P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting consensus metadata
I20260808 13:10:59.792475 8201 ts_tablet_manager.cc:1939] T acc87c7778a5489498fb4c860874ee8a P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting consensus metadata
I20260808 13:10:59.793555 7615 catalog_manager.cc:4841] TS 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337): tablet acc87c7778a5489498fb4c860874ee8a (table test-table [id=5f1be3e2b559439bbacb9f524bab1d86]) successfully deleted
I20260808 13:10:59.793612 7616 catalog_manager.cc:4841] TS 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649): tablet acc87c7778a5489498fb4c860874ee8a (table test-table [id=5f1be3e2b559439bbacb9f524bab1d86]) successfully deleted
I20260808 13:10:59.798198 8202 ts_tablet_manager.cc:1918] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:10:59.798364 8202 log.cc:1196] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/acc87c7778a5489498fb4c860874ee8a
I20260808 13:10:59.798753 8202 ts_tablet_manager.cc:1939] T acc87c7778a5489498fb4c860874ee8a P da81fc3d2bff40f68b2ed1adeea6d329: Deleting consensus metadata
I20260808 13:10:59.799808 7616 catalog_manager.cc:4841] TS da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713): tablet acc87c7778a5489498fb4c860874ee8a (table test-table [id=5f1be3e2b559439bbacb9f524bab1d86]) successfully deleted
Aug 08 13:10:59 dist-test-slave-49h3 krb5kdc[7583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:10:59 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:10:59 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:10:59.859437 8216 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:10:59.848315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.31.59.190:43557)
0808 13:10:59.848535 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.848542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.848881 (+ 339us) server_negotiation.cc:407] Connection header received
0808 13:10:59.849037 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.849043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.849130 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.849262 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:59.850578 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.851723 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.853023 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.853705 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.853837 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.859165 (+ 5328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54620: BlockingRecv error: recv got EOF from 127.0.0.1:54620 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 13:10:59 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM
I20260808 13:10:59.877108 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:10:59.862110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54630 (local address 127.31.59.190:43557)
0808 13:10:59.862305 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:10:59.862310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:10:59.862590 (+ 280us) server_negotiation.cc:407] Connection header received
0808 13:10:59.862769 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:10:59.862772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:10:59.862832 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:10:59.862908 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:10:59.864434 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.865118 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.866948 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:10:59.867338 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:10:59.867426 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:10:59.871949 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:10:59.871954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:10:59.871957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:10:59.871991 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:10:59.874495 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:59.875292 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:59.875296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:59.875299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:59.875436 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:10:59.875844 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:10:59.875847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:10:59.875849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:10:59.876103 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:10:59.876298 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 13:10:59.876610 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 13:10:59.876816 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:10:59.880609 7629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:10:59.881201 7629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:10:59.887939 7923 tablet_service.cc:1467] Processing CreateTablet for tablet f8c48b700d77432984cf32b00d128f5c (DEFAULT_TABLE table=test-table [id=a2e27128ab1c4920bb6d3f33c8868d18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.888116 7772 tablet_service.cc:1467] Processing CreateTablet for tablet f8c48b700d77432984cf32b00d128f5c (DEFAULT_TABLE table=test-table [id=a2e27128ab1c4920bb6d3f33c8868d18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.888137 8073 tablet_service.cc:1467] Processing CreateTablet for tablet f8c48b700d77432984cf32b00d128f5c (DEFAULT_TABLE table=test-table [id=a2e27128ab1c4920bb6d3f33c8868d18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:10:59.888438 7923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c48b700d77432984cf32b00d128f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.888552 8073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c48b700d77432984cf32b00d128f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.888572 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c48b700d77432984cf32b00d128f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:10:59.894893 8175 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Bootstrap starting.
I20260808 13:10:59.895471 8177 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Bootstrap starting.
I20260808 13:10:59.896391 8175 tablet_bootstrap.cc:654] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.897047 8177 tablet_bootstrap.cc:654] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.897425 8176 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Bootstrap starting.
I20260808 13:10:59.899024 8176 tablet_bootstrap.cc:654] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Neither blocks nor log segments found. Creating new log.
I20260808 13:10:59.901219 8175 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: No bootstrap required, opened a new log
I20260808 13:10:59.901378 8175 ts_tablet_manager.cc:1397] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260808 13:10:59.902174 8175 raft_consensus.cc:348] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.902446 8175 raft_consensus.cc:374] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.902539 8175 raft_consensus.cc:729] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7f94c02f0d42abb1fc457cb7e40436, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.902789 8176 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: No bootstrap required, opened a new log
I20260808 13:10:59.902829 8175 consensus_queue.cc:260] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.902992 8176 ts_tablet_manager.cc:1397] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 13:10:59.903966 8176 raft_consensus.cc:348] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.904277 8176 raft_consensus.cc:374] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.905217 8176 raft_consensus.cc:729] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.905369 8177 tablet_bootstrap.cc:492] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: No bootstrap required, opened a new log
I20260808 13:10:59.905550 8177 ts_tablet_manager.cc:1397] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260808 13:10:59.905660 8176 consensus_queue.cc:260] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.906405 8175 ts_tablet_manager.cc:1428] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 13:10:59.906360 8177 raft_consensus.cc:348] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.906596 8177 raft_consensus.cc:374] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:10:59.906689 8177 raft_consensus.cc:729] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Initialized, Role: FOLLOWER
I20260808 13:10:59.906920 8177 consensus_queue.cc:260] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.907498 8177 ts_tablet_manager.cc:1428] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 13:10:59.910398 8176 ts_tablet_manager.cc:1428] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260808 13:10:59.993172 8208 raft_consensus.cc:482] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:10:59.993382 8208 raft_consensus.cc:504] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.994107 8208 leader_election.cc:288] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.994831 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c48b700d77432984cf32b00d128f5c" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:10:59.994859 7944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c48b700d77432984cf32b00d128f5c" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" is_pre_election: true
I20260808 13:10:59.995123 7944 raft_consensus.cc:2454] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.995121 8093 raft_consensus.cc:2454] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 0.
I20260808 13:10:59.995592 7722 leader_election.cc:302] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:10:59.995913 8208 raft_consensus.cc:2781] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:10:59.996024 8208 raft_consensus.cc:482] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:10:59.996110 8208 raft_consensus.cc:3037] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:10:59.998828 8208 raft_consensus.cc:504] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:10:59.999423 8208 leader_election.cc:288] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Requested vote from peers 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:10:59.999975 7944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c48b700d77432984cf32b00d128f5c" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0e53f4a4a4363b77229b52e3c0877"
I20260808 13:10:59.999979 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c48b700d77432984cf32b00d128f5c" candidate_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
I20260808 13:11:00.000202 8093 raft_consensus.cc:3037] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:00.000207 7944 raft_consensus.cc:3037] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:00.003301 8093 raft_consensus.cc:2454] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:11:00.003356 7944 raft_consensus.cc:2454] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da81fc3d2bff40f68b2ed1adeea6d329 in term 1.
I20260808 13:11:00.003765 7722 leader_election.cc:302] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:11:00.004055 8208 raft_consensus.cc:2781] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:00.004220 8208 raft_consensus.cc:686] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Becoming Leader. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Running, Role: LEADER
I20260808 13:11:00.004514 8208 consensus_queue.cc:237] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.007179 7627 catalog_manager.cc:5495] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 reported cstate change: term changed from 0 to 1, leader changed from <none> to da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260808 13:11:00.033655 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.028137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.31.59.129:39713)
0808 13:11:00.028348 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.028352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.028452 (+ 100us) server_negotiation.cc:407] Connection header received
0808 13:11:00.028636 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.028639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.028714 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.028798 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:11:00.029213 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.029998 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.031902 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.032323 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.032411 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.032949 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:11:00.033094 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:11:00.033159 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.033292 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.033357 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:11:00.036567 8093 raft_consensus.cc:1261] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:00.036784 7944 raft_consensus.cc:1261] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Refusing update from remote peer da81fc3d2bff40f68b2ed1adeea6d329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:00.037326 8208 consensus_queue.cc:1035] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:11:00.037672 8181 consensus_queue.cc:1035] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:11:00.049815 7627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:00.050087 7627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:00.055389 7772 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c48b700d77432984cf32b00d128f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 13:11:00.055691 8202 tablet_replica.cc:330] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: stopping tablet replica
I20260808 13:11:00.055863 8202 raft_consensus.cc:2229] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:11:00.056180 8202 raft_consensus.cc:2258] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:00.056298 7627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Sending DeleteTablet for 3 replicas of tablet f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.056880 7923 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c48b700d77432984cf32b00d128f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260808 13:11:00.057514 8201 tablet_replica.cc:330] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: stopping tablet replica
I20260808 13:11:00.057695 8201 raft_consensus.cc:2229] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:00.057927 8201 raft_consensus.cc:2258] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:00.058243 8073 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c48b700d77432984cf32b00d128f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36496
I20260808 13:11:00.058543 8200 tablet_replica.cc:330] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: stopping tablet replica
I20260808 13:11:00.058336 7627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:11:00.058710 8200 raft_consensus.cc:2229] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:00.058935 8200 raft_consensus.cc:2258] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:11:00.058977 7627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:11:00.059476 7923 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c48b700d77432984cf32b00d128f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260808 13:11:00.059669 7772 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c48b700d77432984cf32b00d128f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 13:11:00.060734 8200 ts_tablet_manager.cc:1905] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.062062 8202 ts_tablet_manager.cc:1905] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.068945 8202 ts_tablet_manager.cc:1918] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:00.069164 8202 log.cc:1196] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.069521 8073 tablet_service.cc:1467] Processing CreateTablet for tablet a179f40731ef4444a33e096bb9572ff2 (DEFAULT_TABLE table=test-table [id=2d291ce9b661451db44e0f13fc1459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:00.070011 8073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a179f40731ef4444a33e096bb9572ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:00.070034 8202 ts_tablet_manager.cc:1939] T f8c48b700d77432984cf32b00d128f5c P da81fc3d2bff40f68b2ed1adeea6d329: Deleting consensus metadata
W20260808 13:11:00.071863 7616 catalog_manager.cc:4808] TS da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713): delete failed for tablet f8c48b700d77432984cf32b00d128f5c because the tablet was not found. No further retry: Not found: Tablet not found: f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.072254 8201 ts_tablet_manager.cc:1905] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.073850 7923 tablet_service.cc:1467] Processing CreateTablet for tablet a179f40731ef4444a33e096bb9572ff2 (DEFAULT_TABLE table=test-table [id=2d291ce9b661451db44e0f13fc1459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:00.074182 7923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a179f40731ef4444a33e096bb9572ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:00.079253 8201 ts_tablet_manager.cc:1918] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:00.079351 8177 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Bootstrap starting.
I20260808 13:11:00.079493 8201 log.cc:1196] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.079435 7772 tablet_service.cc:1467] Processing CreateTablet for tablet a179f40731ef4444a33e096bb9572ff2 (DEFAULT_TABLE table=test-table [id=2d291ce9b661451db44e0f13fc1459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:00.079859 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a179f40731ef4444a33e096bb9572ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:00.080042 8201 ts_tablet_manager.cc:1939] T f8c48b700d77432984cf32b00d128f5c P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting consensus metadata
I20260808 13:11:00.081067 8177 tablet_bootstrap.cc:654] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Neither blocks nor log segments found. Creating new log.
W20260808 13:11:00.082508 7616 catalog_manager.cc:4808] TS 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649): delete failed for tablet f8c48b700d77432984cf32b00d128f5c because the tablet was not found. No further retry: Not found: Tablet not found: f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.086295 8200 ts_tablet_manager.cc:1918] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:00.086381 8176 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Bootstrap starting.
I20260808 13:11:00.088300 8176 tablet_bootstrap.cc:654] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:00.088677 8175 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Bootstrap starting.
I20260808 13:11:00.090531 8175 tablet_bootstrap.cc:654] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:00.088698 8200 log.cc:1196] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/f8c48b700d77432984cf32b00d128f5c
I20260808 13:11:00.094249 8200 ts_tablet_manager.cc:1939] T f8c48b700d77432984cf32b00d128f5c P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting consensus metadata
I20260808 13:11:00.094887 8176 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: No bootstrap required, opened a new log
I20260808 13:11:00.095106 8176 ts_tablet_manager.cc:1397] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260808 13:11:00.095422 7615 catalog_manager.cc:4841] TS 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337): tablet f8c48b700d77432984cf32b00d128f5c (table test-table [id=a2e27128ab1c4920bb6d3f33c8868d18]) successfully deleted
I20260808 13:11:00.096653 8175 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: No bootstrap required, opened a new log
I20260808 13:11:00.096827 8175 ts_tablet_manager.cc:1397] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 13:11:00.097376 8176 raft_consensus.cc:348] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.097716 8176 raft_consensus.cc:374] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:00.097905 8176 raft_consensus.cc:729] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da81fc3d2bff40f68b2ed1adeea6d329, State: Initialized, Role: FOLLOWER
I20260808 13:11:00.097402 8177 tablet_bootstrap.cc:492] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: No bootstrap required, opened a new log
I20260808 13:11:00.098269 8176 consensus_queue.cc:260] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.098613 8177 ts_tablet_manager.cc:1397] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.001s
I20260808 13:11:00.099000 8176 ts_tablet_manager.cc:1428] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 13:11:00.098997 8175 raft_consensus.cc:348] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.099269 8175 raft_consensus.cc:374] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:00.099346 8175 raft_consensus.cc:729] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7f94c02f0d42abb1fc457cb7e40436, State: Initialized, Role: FOLLOWER
I20260808 13:11:00.099491 8177 raft_consensus.cc:348] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.099773 8177 raft_consensus.cc:374] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:00.099618 8175 consensus_queue.cc:260] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.099854 8177 raft_consensus.cc:729] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Initialized, Role: FOLLOWER
I20260808 13:11:00.100158 8177 consensus_queue.cc:260] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.100677 8177 ts_tablet_manager.cc:1428] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 13:11:00.101492 8175 ts_tablet_manager.cc:1428] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260808 13:11:00.195623 8184 raft_consensus.cc:482] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:11:00.195835 8184 raft_consensus.cc:504] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.196496 8184 leader_election.cc:288] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:11:00.197106 7794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a179f40731ef4444a33e096bb9572ff2" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" is_pre_election: true
I20260808 13:11:00.197130 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a179f40731ef4444a33e096bb9572ff2" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" is_pre_election: true
I20260808 13:11:00.197343 7794 raft_consensus.cc:2454] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 0.
I20260808 13:11:00.197404 8093 raft_consensus.cc:2454] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 0.
I20260808 13:11:00.197813 7874 leader_election.cc:302] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:11:00.198101 8184 raft_consensus.cc:2781] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:11:00.198216 8184 raft_consensus.cc:482] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:11:00.198292 8184 raft_consensus.cc:3037] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:00.200356 8184 raft_consensus.cc:504] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.200891 8184 leader_election.cc:288] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 election: Requested vote from peers da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713), 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337)
I20260808 13:11:00.201464 7794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a179f40731ef4444a33e096bb9572ff2" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da81fc3d2bff40f68b2ed1adeea6d329"
I20260808 13:11:00.201489 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a179f40731ef4444a33e096bb9572ff2" candidate_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7f94c02f0d42abb1fc457cb7e40436"
I20260808 13:11:00.201678 8093 raft_consensus.cc:3037] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:00.201687 7794 raft_consensus.cc:3037] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:00.203764 7794 raft_consensus.cc:2454] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 1.
I20260808 13:11:00.203778 8093 raft_consensus.cc:2454] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab0e53f4a4a4363b77229b52e3c0877 in term 1.
I20260808 13:11:00.204195 7873 leader_election.cc:302] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c7f94c02f0d42abb1fc457cb7e40436, 6ab0e53f4a4a4363b77229b52e3c0877, da81fc3d2bff40f68b2ed1adeea6d329; no voters:
I20260808 13:11:00.204514 8184 raft_consensus.cc:2781] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:00.204643 8184 raft_consensus.cc:686] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 LEADER]: Becoming Leader. State: Replica: 6ab0e53f4a4a4363b77229b52e3c0877, State: Running, Role: LEADER
I20260808 13:11:00.204924 8184 consensus_queue.cc:237] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } }
I20260808 13:11:00.207474 7630 catalog_manager.cc:5495] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ab0e53f4a4a4363b77229b52e3c0877" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 35649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260808 13:11:00.229331 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.224741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.31.59.130:35649)
0808 13:11:00.225183 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.225187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.225202 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:00.225261 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.225264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.225333 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.225413 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:11:00.225845 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.226641 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.227858 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.228255 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.228339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.228684 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:11:00.228785 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:11:00.228844 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.228936 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.228999 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260808 13:11:00.241137 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.237106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:45855 (local address 127.31.59.130:35649)
0808 13:11:00.237234 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.237238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.237282 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:11:00.237418 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.237421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.237509 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.237575 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:11:00.237904 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.238620 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.239802 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.240186 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.240276 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.240587 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:11:00.240692 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:11:00.240753 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.240843 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.240898 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":41}
I20260808 13:11:00.241212 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.237037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:35649 (local address 127.31.59.130:45855)
0808 13:11:00.237223 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:00.237239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:00.237332 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:00.237601 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:00.237605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:00.237613 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:11:00.237784 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:00.237791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.238726 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.238729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:00.239686 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:00.239697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.240294 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.240298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:00.240417 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.240458 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:11:00.240768 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:11:00.240776 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:11:00.240829 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 13:11:00.240928 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":16}
I20260808 13:11:00.248769 8093 raft_consensus.cc:1261] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Refusing update from remote peer 6ab0e53f4a4a4363b77229b52e3c0877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:00.248787 7794 raft_consensus.cc:1261] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Refusing update from remote peer 6ab0e53f4a4a4363b77229b52e3c0877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:00.249300 8220 consensus_queue.cc:1035] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da81fc3d2bff40f68b2ed1adeea6d329" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:11:00.249639 8236 consensus_queue.cc:1035] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7f94c02f0d42abb1fc457cb7e40436" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:11:00.293605 7630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:00.293840 7630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:00.296784 7630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f8ad272b3a241079ddb0c23be355a8e: Sending DeleteTablet for 3 replicas of tablet a179f40731ef4444a33e096bb9572ff2
I20260808 13:11:00.297704 7923 tablet_service.cc:1514] Processing DeleteTablet for tablet a179f40731ef4444a33e096bb9572ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260808 13:11:00.298049 8201 tablet_replica.cc:330] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: stopping tablet replica
I20260808 13:11:00.298034 7772 tablet_service.cc:1514] Processing DeleteTablet for tablet a179f40731ef4444a33e096bb9572ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 13:11:00.298228 8201 raft_consensus.cc:2229] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:11:00.298347 8202 tablet_replica.cc:330] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: stopping tablet replica
I20260808 13:11:00.298370 8073 tablet_service.cc:1514] Processing DeleteTablet for tablet a179f40731ef4444a33e096bb9572ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:00 UTC) from {username='oryx', principal='oryx/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36496
I20260808 13:11:00.298537 8202 raft_consensus.cc:2229] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:00.298736 8201 raft_consensus.cc:2258] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:00.298724 8202 raft_consensus.cc:2258] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:00.298810 8200 tablet_replica.cc:330] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: stopping tablet replica
I20260808 13:11:00.299033 8200 raft_consensus.cc:2229] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:00.299227 8200 raft_consensus.cc:2258] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:00.299938 8202 ts_tablet_manager.cc:1905] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.300624 8200 ts_tablet_manager.cc:1905] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.304937 8202 ts_tablet_manager.cc:1918] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:11:00.305073 8202 log.cc:1196] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/a179f40731ef4444a33e096bb9572ff2
I20260808 13:11:00.305400 8202 ts_tablet_manager.cc:1939] T a179f40731ef4444a33e096bb9572ff2 P da81fc3d2bff40f68b2ed1adeea6d329: Deleting consensus metadata
I20260808 13:11:00.306025 8201 ts_tablet_manager.cc:1905] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:00.307648 8200 ts_tablet_manager.cc:1918] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:11:00.307821 8200 log.cc:1196] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/a179f40731ef4444a33e096bb9572ff2
I20260808 13:11:00.308449 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 7702
I20260808 13:11:00.310149 7616 catalog_manager.cc:4841] TS da81fc3d2bff40f68b2ed1adeea6d329 (127.31.59.129:39713): tablet a179f40731ef4444a33e096bb9572ff2 (table test-table [id=2d291ce9b661451db44e0f13fc1459eb]) successfully deleted
I20260808 13:11:00.312188 8201 ts_tablet_manager.cc:1918] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:11:00.312342 8201 log.cc:1196] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/a179f40731ef4444a33e096bb9572ff2
I20260808 13:11:00.312693 8201 ts_tablet_manager.cc:1939] T a179f40731ef4444a33e096bb9572ff2 P 6ab0e53f4a4a4363b77229b52e3c0877: Deleting consensus metadata
I20260808 13:11:00.313613 7616 catalog_manager.cc:4841] TS 6ab0e53f4a4a4363b77229b52e3c0877 (127.31.59.130:35649): tablet a179f40731ef4444a33e096bb9572ff2 (table test-table [id=2d291ce9b661451db44e0f13fc1459eb]) successfully deleted
I20260808 13:11:00.313863 8200 ts_tablet_manager.cc:1939] T a179f40731ef4444a33e096bb9572ff2 P 1c7f94c02f0d42abb1fc457cb7e40436: Deleting consensus metadata
I20260808 13:11:00.314818 7615 catalog_manager.cc:4841] TS 1c7f94c02f0d42abb1fc457cb7e40436 (127.31.59.131:39337): tablet a179f40731ef4444a33e096bb9572ff2 (table test-table [id=2d291ce9b661451db44e0f13fc1459eb]) successfully deleted
Aug 08 13:11:00 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.129@KRBTEST.COM
I20260808 13:11:00.328711 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.309663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.31.59.129:39713)
0808 13:11:00.309852 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.309858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.315057 (+ 5199us) server_negotiation.cc:407] Connection header received
0808 13:11:00.315230 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.315234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.315309 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.315389 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:00.316618 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.317367 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.318761 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.319185 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.319307 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.323156 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:00.323172 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:00.323187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:00.323230 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:00.326107 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.326729 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.326736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.326738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.326876 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.327234 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.327237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.327239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.327600 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:00.327935 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.328183 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.328406 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:11:00.329541 7843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:00.458012 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 7702
I20260808 13:11:00.485657 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 7853
Aug 08 13:11:00 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.130@KRBTEST.COM
I20260808 13:11:00.501575 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.486440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.31.59.130:35649)
0808 13:11:00.486621 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.486626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.486720 (+ 94us) server_negotiation.cc:407] Connection header received
0808 13:11:00.486910 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.486915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.486981 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.487063 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:00.488511 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.489346 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.490629 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.491114 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.491230 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.495408 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:00.495427 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:00.495442 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:00.495480 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:00.498703 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.499384 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.499391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.499394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.499575 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.499927 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.499933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.499936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.500333 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:00.500689 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.500942 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.501170 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:11:00.502236 7995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:00.653956 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 7853
I20260808 13:11:00.680794 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8004
Aug 08 13:11:00 dist-test-slave-49h3 krb5kdc[7583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.131@KRBTEST.COM
I20260808 13:11:00.695806 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:00.681612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.31.59.131:39337)
0808 13:11:00.682100 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:00.682104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:00.682119 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:00.682175 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:00.682179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:00.682239 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:00.682335 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:00.683646 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.684382 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.685678 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:00.686120 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:00.686233 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:00.690471 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:00.690490 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:00.690507 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:00.690549 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:00.693332 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.694003 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.694010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.694012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.694164 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:00.694448 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:00.694450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:00.694452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:00.694828 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:00.695046 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:00.695254 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 13:11:00.695450 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":261,"threads_started":1}
I20260808 13:11:00.696436 8142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:00.797899 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8004
I20260808 13:11:00.817710 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 7596
I20260808 13:11:00.818994 7671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:00.938848 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 7596
2026-08-08T13:11:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6763 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:11:01 dist-test-slave-49h3 krb5kdc[8275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:11:01 dist-test-slave-49h3 krb5kdc[8275](info): set up 2 sockets
Aug 08 13:11:01 dist-test-slave-49h3 krb5kdc[8275](info): commencing operation
krb5kdc: starting...
W20260808 13:11:02.035737 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:11:02 dist-test-slave-49h3 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:11:02.298821 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45023
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:02.628427 8289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:02.628731 8289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:02.628865 8289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:02.636224 8289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:02.636348 8289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:02.636438 8289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:02.636519 8289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:02.636583 8289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:02.646726 8289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:02.648084 8289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:02.649416 8289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:02.661882 8296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:02.661954 8298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:02.661767 8294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:02.663800 8289 server_base.cc:1034] running on GCE node
I20260808 13:11:02.664786 8289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:02.666534 8289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:02.667763 8289 hybrid_clock.cc:648] HybridClock initialized: now 1786194662667696 us; error 68 us; skew 500 ppm
I20260808 13:11:02.668126 8289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T13:11:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1798 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:11:02 dist-test-slave-49h3 krb5kdc[8304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:11:02 dist-test-slave-49h3 krb5kdc[8304](info): set up 2 sockets
Aug 08 13:11:02 dist-test-slave-49h3 krb5kdc[8304](info): commencing operation
krb5kdc: starting...
W20260808 13:11:03.813481 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:11:03 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:11:04.079027 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38483
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:04.412331 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:04.412628 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:04.412774 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:04.420038 8317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:04.420171 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:04.420253 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:04.420331 8317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:04.420392 8317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:04.430548 8317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:04.431922 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:04.433270 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:04.444590 8323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:04.444568 8327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:04.444579 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:04.445494 8317 server_base.cc:1034] running on GCE node
I20260808 13:11:04.446287 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:04.447670 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:04.448877 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786194664448823 us; error 40 us; skew 500 ppm
I20260808 13:11:04.449208 8317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:04 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194664, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:04.452173 8317 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:11:04.453644 8317 webserver.cc:460] Webserver started at http://127.31.59.190:46323/ using document root <none> and password file <none>
I20260808 13:11:04.454309 8317 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:04.454425 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:04.454723 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:04.457738 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "33ab827f5f434dd782bfc4e67c989e17"
format_stamp: "Formatted at 2026-08-08 13:11:04 on dist-test-slave-49h3"
server_key: "72e5be46b10991d2f8029abee6640e41"
server_key_iv: "ca8ff9000f47dc869e1f0b23eef01a6c"
server_key_version: "encryptionkey@0"
I20260808 13:11:04.458478 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "33ab827f5f434dd782bfc4e67c989e17"
format_stamp: "Formatted at 2026-08-08 13:11:04 on dist-test-slave-49h3"
server_key: "72e5be46b10991d2f8029abee6640e41"
server_key_iv: "ca8ff9000f47dc869e1f0b23eef01a6c"
server_key_version: "encryptionkey@0"
I20260808 13:11:04.463574 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:11:04.466920 8335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:04.468147 8317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:11:04.468310 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "33ab827f5f434dd782bfc4e67c989e17"
format_stamp: "Formatted at 2026-08-08 13:11:04 on dist-test-slave-49h3"
server_key: "72e5be46b10991d2f8029abee6640e41"
server_key_iv: "ca8ff9000f47dc869e1f0b23eef01a6c"
server_key_version: "encryptionkey@0"
I20260808 13:11:04.468487 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:04.489368 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:04.493266 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:04.493572 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:04.511351 8317 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:38483
I20260808 13:11:04.511389 8388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:38483 every 8 connection(s)
I20260808 13:11:04.512759 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:11:04.513808 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8317
I20260808 13:11:04.514058 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:11:04.514854 31982 external_mini_cluster.cc:1428] Setting key 58cf946c9b23bbf8d228b094cc4e246b
I20260808 13:11:04.517529 8389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:11:04 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:04.528389 8389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Bootstrap starting.
I20260808 13:11:04.531762 8389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:04.532809 8389 log.cc:824] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:04.535651 8389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: No bootstrap required, opened a new log
I20260808 13:11:04.536509 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:04.516964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.31.59.190:38483)
0808 13:11:04.517827 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:04.517838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:04.517878 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:11:04.519556 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:04.519576 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:04.519945 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:04.520291 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:04.521802 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:04.523130 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:04.524395 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:04.525091 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:04.525288 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:04.529405 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:04.529413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:04.529426 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:04.529498 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:04.532491 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:04.533156 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:04.533162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:04.533169 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:04.533392 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:04.533819 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:04.533823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:04.533827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:04.534427 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:04.534706 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:04.535147 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 13:11:04.535537 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":364,"threads_started":1}
I20260808 13:11:04.543716 8389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:04.544152 8389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:04.544269 8389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ab827f5f434dd782bfc4e67c989e17, State: Initialized, Role: FOLLOWER
I20260808 13:11:04.544902 8389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:04.545152 8389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:11:04.545287 8389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:11:04.545461 8389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:04.547704 8389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:04.548213 8389 leader_election.cc:302] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33ab827f5f434dd782bfc4e67c989e17; no voters:
I20260808 13:11:04.548910 8389 leader_election.cc:288] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:11:04.549015 8394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:04.549728 8394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 LEADER]: Becoming Leader. State: Replica: 33ab827f5f434dd782bfc4e67c989e17, State: Running, Role: LEADER
I20260808 13:11:04.550302 8394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:04.550990 8389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:11:04.552781 8395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ab827f5f434dd782bfc4e67c989e17. Latest consensus state: current_term: 1 leader_uuid: "33ab827f5f434dd782bfc4e67c989e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } } }
I20260808 13:11:04.553211 8395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:04.554842 8402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:11:04.552809 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33ab827f5f434dd782bfc4e67c989e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } } }
I20260808 13:11:04.554984 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:04.560159 8402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:11:04.569059 8402 catalog_manager.cc:1347] Generated new cluster ID: 208274c6fb5f4652bdb114c01d484dff
I20260808 13:11:04.569182 8402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:11:04.600142 8402 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:11:04.602101 8402 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:11:04.610436 8402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Generated new TSK 0
I20260808 13:11:04.611459 8402 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:11:04.681042 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:05.020887 8416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:05.021216 8416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:05.021359 8416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:05.028793 8416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:05.028925 8416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:05.029076 8416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:11:05.039191 8416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:05.040627 8416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:05.042137 8416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:05.054996 8421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:05.055030 8422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:05.055250 8424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:05.055944 8416 server_base.cc:1034] running on GCE node
I20260808 13:11:05.056576 8416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:05.057585 8416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:05.058772 8416 hybrid_clock.cc:648] HybridClock initialized: now 1786194665058707 us; error 60 us; skew 500 ppm
I20260808 13:11:05.059062 8416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:05 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194665, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:05.062225 8416 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:11:05.063589 8416 webserver.cc:460] Webserver started at http://127.31.59.129:39269/ using document root <none> and password file <none>
I20260808 13:11:05.064268 8416 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:05.064385 8416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:05.064673 8416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:05.067651 8416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "80bfdb024976427bb90cc486ba0b0f49"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "ae0b465080e564cfb71d0de1a3bedb3e"
server_key_iv: "265a1c893e39141ff9ba5c42fd6a38bf"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.068423 8416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "80bfdb024976427bb90cc486ba0b0f49"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "ae0b465080e564cfb71d0de1a3bedb3e"
server_key_iv: "265a1c893e39141ff9ba5c42fd6a38bf"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.073642 8416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:11:05.076958 8432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.078233 8416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:11:05.078378 8416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "80bfdb024976427bb90cc486ba0b0f49"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "ae0b465080e564cfb71d0de1a3bedb3e"
server_key_iv: "265a1c893e39141ff9ba5c42fd6a38bf"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.078552 8416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:05.092128 8416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:05.095947 8416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:05.096220 8416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:05.097245 8416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:05.098639 8416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:05.098717 8416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.098845 8416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:05.098907 8416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.130015 8416 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:38829
I20260808 13:11:05.130080 8550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:38829 every 8 connection(s)
I20260808 13:11:05.131551 8416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:11:05.138664 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8416
I20260808 13:11:05.138876 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:11:05.139467 31982 external_mini_cluster.cc:1428] Setting key 84216c7aaacf4ee59d3727cb8994f114
Aug 08 13:11:05 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194665, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:05.152684 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:05.134630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:44105 (local address 127.31.59.190:38483)
0808 13:11:05.135099 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:05.135105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:05.136197 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 13:11:05.138076 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:05.138080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:05.138151 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:05.138266 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:05.139768 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.140632 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.142330 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.142888 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.143013 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:05.146444 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:05.146450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:05.146454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:05.146488 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:05.149488 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:05.150179 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:05.150184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:05.150187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:05.150338 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:05.150900 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:05.150905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:05.150907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:05.151275 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:05.151408 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:05.152110 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 13:11:05.152330 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":215,"threads_started":1}
I20260808 13:11:05.153796 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:05.135237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:44105)
0808 13:11:05.136042 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:05.136081 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:05.137839 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:05.138435 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:05.138443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:05.139004 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:05.139590 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:05.139614 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.140779 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.140783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:05.142171 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:05.142194 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.143105 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.143110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:05.143347 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:05.144007 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:05.144030 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:05.146234 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:05.149622 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:05.149629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:05.149642 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:05.150055 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:05.150451 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:05.150456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:05.150460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:05.150747 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:05.151385 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:05.151390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:05.151719 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 13:11:05.151952 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 13:11:05.152406 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":350,"threads_started":1}
I20260808 13:11:05.155290 8551 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38483
I20260808 13:11:05.155629 8551 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:05.156523 8551 heartbeater.cc:511] Master 127.31.59.190:38483 requested a full tablet report, sending...
I20260808 13:11:05.158771 8353 ts_manager.cc:194] Registered new tserver with Master: 80bfdb024976427bb90cc486ba0b0f49 (127.31.59.129:38829)
I20260808 13:11:05.160223 8353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:44105
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:11:05.198665 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:05.529047 8557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:05.529366 8557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:05.529539 8557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:05.536984 8557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:05.537113 8557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:05.537290 8557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:11:05.547382 8557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:05.548732 8557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:05.550064 8557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:05.562774 8563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:05.563045 8562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:05.563195 8565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:05.563474 8557 server_base.cc:1034] running on GCE node
I20260808 13:11:05.564069 8557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:05.564958 8557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:05.566154 8557 hybrid_clock.cc:648] HybridClock initialized: now 1786194665566102 us; error 45 us; skew 500 ppm
I20260808 13:11:05.566452 8557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:05 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194665, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:05.569429 8557 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:11:05.570955 8557 webserver.cc:460] Webserver started at http://127.31.59.130:37153/ using document root <none> and password file <none>
I20260808 13:11:05.571659 8557 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:05.571780 8557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:05.572105 8557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:05.575029 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "e7e8560035e34253af9ff6aedae32abc"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "69884fbd7a5bcfda16be17e5d67964f8"
server_key_iv: "608b3f3e46e5a8fcfebce4d951908a61"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.575767 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "e7e8560035e34253af9ff6aedae32abc"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "69884fbd7a5bcfda16be17e5d67964f8"
server_key_iv: "608b3f3e46e5a8fcfebce4d951908a61"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.580750 8557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:11:05.584179 8572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.585510 8557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:11:05.585670 8557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "e7e8560035e34253af9ff6aedae32abc"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "69884fbd7a5bcfda16be17e5d67964f8"
server_key_iv: "608b3f3e46e5a8fcfebce4d951908a61"
server_key_version: "encryptionkey@0"
I20260808 13:11:05.585857 8557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:05.603518 8557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:05.607388 8557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:05.607659 8557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:05.608667 8557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:05.610070 8557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:05.610153 8557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.610286 8557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:05.610340 8557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:05.641851 8557 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34933
I20260808 13:11:05.641891 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34933 every 8 connection(s)
I20260808 13:11:05.643146 8557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:11:05.643527 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8557
I20260808 13:11:05.643711 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:11:05.644263 31982 external_mini_cluster.cc:1428] Setting key 43a265975071e5f03c943dcffc534ed2
Aug 08 13:11:05 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194665, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:05.663897 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:05.645793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:39543 (local address 127.31.59.190:38483)
0808 13:11:05.645956 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:05.645960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:05.647655 (+ 1695us) server_negotiation.cc:407] Connection header received
0808 13:11:05.649552 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:05.649557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:05.649640 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:05.649742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:05.651458 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.652424 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.654001 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.654598 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.654722 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:05.658371 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:05.658376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:05.658380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:05.658413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:05.660894 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:05.661609 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:05.661614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:05.661618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:05.661757 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:05.662175 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:05.662178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:05.662180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:05.662466 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:05.662601 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:05.663298 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 13:11:05.663537 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 13:11:05.664908 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:05.646557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:39543)
0808 13:11:05.647463 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:05.647515 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:05.649252 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:05.649938 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:05.649946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:05.650534 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:05.651290 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:05.651310 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.652573 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.652577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:05.653868 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:05.653880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:05.654807 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:05.654813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:05.655010 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:05.655665 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:05.655692 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:05.658142 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:05.661028 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:05.661037 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:05.661054 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:05.661499 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:05.661864 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:05.661869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:05.661873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:05.662063 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:05.662593 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:05.662600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:05.662938 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 13:11:05.663174 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 13:11:05.663704 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":370,"threads_started":1}
I20260808 13:11:05.666493 8691 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38483
I20260808 13:11:05.666801 8691 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:05.667515 8691 heartbeater.cc:511] Master 127.31.59.190:38483 requested a full tablet report, sending...
I20260808 13:11:05.668965 8353 ts_manager.cc:194] Registered new tserver with Master: e7e8560035e34253af9ff6aedae32abc (127.31.59.130:34933)
I20260808 13:11:05.669785 8353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:39543
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:11:05.700453 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:06.029699 8696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:06.030005 8696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:06.030149 8696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:06.037348 8696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:06.037495 8696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:06.037650 8696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:11:06.047761 8696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:06.049093 8696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:06.050462 8696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:06.063380 8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:06.063334 8707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:06.063405 8704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:06.064296 8696 server_base.cc:1034] running on GCE node
I20260808 13:11:06.064867 8696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:06.065776 8696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:06.066970 8696 hybrid_clock.cc:648] HybridClock initialized: now 1786194666066914 us; error 46 us; skew 500 ppm
I20260808 13:11:06.067257 8696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:06 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194666, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:06.073906 8696 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:11:06.075369 8696 webserver.cc:460] Webserver started at http://127.31.59.131:41037/ using document root <none> and password file <none>
I20260808 13:11:06.076049 8696 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:06.076165 8696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:06.076452 8696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:06.079348 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "e6e79c2222e547cd8c4abb7bbdb9b611"
format_stamp: "Formatted at 2026-08-08 13:11:06 on dist-test-slave-49h3"
server_key: "a33a23ce00dc9d659bfdc74c904e6104"
server_key_iv: "19a2204410d53a70fb556d481e6b966d"
server_key_version: "encryptionkey@0"
I20260808 13:11:06.080116 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "e6e79c2222e547cd8c4abb7bbdb9b611"
format_stamp: "Formatted at 2026-08-08 13:11:06 on dist-test-slave-49h3"
server_key: "a33a23ce00dc9d659bfdc74c904e6104"
server_key_iv: "19a2204410d53a70fb556d481e6b966d"
server_key_version: "encryptionkey@0"
I20260808 13:11:06.085233 8696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:11:06.088469 8714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:06.089838 8696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 13:11:06.089982 8696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "e6e79c2222e547cd8c4abb7bbdb9b611"
format_stamp: "Formatted at 2026-08-08 13:11:06 on dist-test-slave-49h3"
server_key: "a33a23ce00dc9d659bfdc74c904e6104"
server_key_iv: "19a2204410d53a70fb556d481e6b966d"
server_key_version: "encryptionkey@0"
I20260808 13:11:06.090152 8696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:06.110061 8696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:06.114341 8696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:06.114645 8696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:06.115715 8696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:06.117097 8696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:06.117178 8696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:06.117306 8696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:06.117362 8696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:06.150975 8696 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:33379
I20260808 13:11:06.151026 8834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:33379 every 8 connection(s)
I20260808 13:11:06.152316 8696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:11:06.155721 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8696
I20260808 13:11:06.155946 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:11:06.156415 31982 external_mini_cluster.cc:1428] Setting key 891009e42af6b74fb1d7ed66ba644b2e
I20260808 13:11:06.163637 8551 heartbeater.cc:503] Master 127.31.59.190:38483 was elected leader, sending a full tablet report...
Aug 08 13:11:06 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194666, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:06.172518 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:06.154875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:42593 (local address 127.31.59.190:38483)
0808 13:11:06.155057 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:06.155062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:06.156317 (+ 1255us) server_negotiation.cc:407] Connection header received
0808 13:11:06.158406 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:06.158411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:06.158495 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:06.158589 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:06.160231 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.160978 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.162502 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.162917 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.163026 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:06.166802 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:06.166807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:06.166810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:06.166844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:06.169224 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.170117 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.170122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.170124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.170256 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.170678 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.170682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.170684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.170979 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:06.171115 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:06.171885 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 13:11:06.172148 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:11:06.173497 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:06.155320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:42593)
0808 13:11:06.156136 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:06.156192 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:06.158070 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:06.158866 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:06.158875 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:06.159500 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:06.160066 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:06.160084 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.161120 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.161124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:06.162368 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:06.162380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.163146 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.163153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:06.163402 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:06.164119 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:06.164146 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:06.166590 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:06.169355 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:06.169362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:06.169377 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:06.169990 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:06.170365 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:06.170370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:06.170374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:06.170563 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:06.171104 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:06.171112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:06.171487 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 13:11:06.171747 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 13:11:06.172308 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":352,"threads_started":1}
I20260808 13:11:06.174758 8835 heartbeater.cc:347] Connected to a master server at 127.31.59.190:38483
I20260808 13:11:06.175053 8835 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:06.175765 8835 heartbeater.cc:511] Master 127.31.59.190:38483 requested a full tablet report, sending...
I20260808 13:11:06.177090 8353 ts_manager.cc:194] Registered new tserver with Master: e6e79c2222e547cd8c4abb7bbdb9b611 (127.31.59.131:33379)
I20260808 13:11:06.177956 8353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:42593
I20260808 13:11:06.183871 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
Entry for principal oryx/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab.
WARNING: no policy specified for oryx/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
Entry for principal oryx/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab.
WARNING: no policy specified for oryx/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
Entry for principal oryx/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab.
I20260808 13:11:06.347703 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8416
Aug 08 13:11:06 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:11:06.363556 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:06.348934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.31.59.129:38829)
0808 13:11:06.349636 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:06.349643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:06.349669 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:11:06.349732 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:06.349739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:06.349965 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:06.350092 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:06.351432 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.352521 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.353810 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.354367 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.354460 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:06.358505 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:06.358518 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:06.358532 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:06.358570 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:06.361161 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.361831 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.361837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.361839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.361964 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.362243 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.362246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.362249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.362576 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:06.362821 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:06.362991 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:11:06.363196 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":357,"threads_started":1}
I20260808 13:11:06.364795 8546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:06.447755 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8416
I20260808 13:11:06.467089 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8557
Aug 08 13:11:06 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:11:06.482489 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:06.467943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.31.59.130:34933)
0808 13:11:06.468478 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:06.468486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:06.468510 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:11:06.468575 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:06.468582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:06.468781 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:06.468926 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:06.470179 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.471175 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.472431 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.472963 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.473074 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:06.477081 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:06.477093 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:06.477108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:06.477143 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:06.479790 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.480403 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.480410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.480412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.480538 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.480882 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.480886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.480888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.481236 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:06.481558 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:06.481876 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 13:11:06.482145 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":239,"threads_started":1}
I20260808 13:11:06.483747 8686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:06.567277 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8557
I20260808 13:11:06.585858 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8696
Aug 08 13:11:06 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:11:06.601655 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:06.586949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56732 (local address 127.31.59.131:33379)
0808 13:11:06.587599 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:06.587606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:06.587634 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:11:06.587717 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:06.587723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:06.587951 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:06.588080 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:06.589348 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.590331 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.591488 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:06.592015 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:06.592103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:06.596511 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:06.596523 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:06.596536 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:06.596577 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:06.599336 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.599882 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.599887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.599889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.600013 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:06.600293 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:06.600296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:06.600298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:06.600636 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:06.600866 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:06.601065 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 13:11:06.601263 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":346,"threads_started":1}
I20260808 13:11:06.602936 8830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:06.684521 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8696
I20260808 13:11:06.701834 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8317
I20260808 13:11:06.703059 8384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:06.810137 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8317
I20260808 13:11:06.832355 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38483
--webserver_interface=127.31.59.190
--webserver_port=46323
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:07.166736 8852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:07.167042 8852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:07.167191 8852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:07.175038 8852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:07.175156 8852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:07.175239 8852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:07.175312 8852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:07.175371 8852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:07.185410 8852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=46323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:07.186774 8852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:07.188128 8852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:07.199792 8857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:07.199716 8858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:07.199769 8860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:07.201009 8852 server_base.cc:1034] running on GCE node
I20260808 13:11:07.201829 8852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:07.203264 8852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:07.204476 8852 hybrid_clock.cc:648] HybridClock initialized: now 1786194667204411 us; error 48 us; skew 500 ppm
I20260808 13:11:07.204829 8852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:07 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:07.207964 8852 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:11:07.209470 8852 webserver.cc:460] Webserver started at http://127.31.59.190:46323/ using document root <none> and password file <none>
I20260808 13:11:07.210156 8852 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:07.210270 8852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:07.215698 8852 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:11:07.218428 8869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:07.219753 8852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:11:07.219907 8852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "33ab827f5f434dd782bfc4e67c989e17"
format_stamp: "Formatted at 2026-08-08 13:11:04 on dist-test-slave-49h3"
server_key: "72e5be46b10991d2f8029abee6640e41"
server_key_iv: "ca8ff9000f47dc869e1f0b23eef01a6c"
server_key_version: "encryptionkey@0"
I20260808 13:11:07.220887 8852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:07.244997 8852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:07.249166 8852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:07.249420 8852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:07.267380 8852 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:38483
I20260808 13:11:07.267382 8925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:38483 every 8 connection(s)
I20260808 13:11:07.268918 8852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:11:07.275673 8926 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:11:07.276420 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8852
I20260808 13:11:07.277367 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:38829
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:38483
--webserver_port=39269
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:11:07.280198 8926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Bootstrap starting.
I20260808 13:11:07.301855 8926 log.cc:824] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:07.311009 8926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:11:07.311537 8926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Bootstrap complete.
I20260808 13:11:07.320676 8926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:07.321569 8926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ab827f5f434dd782bfc4e67c989e17, State: Initialized, Role: FOLLOWER
I20260808 13:11:07.322206 8926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:07.322438 8926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:11:07.322551 8926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:11:07.322716 8926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:11:07.326906 8926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:07.327418 8926 leader_election.cc:302] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33ab827f5f434dd782bfc4e67c989e17; no voters:
I20260808 13:11:07.328028 8926 leader_election.cc:288] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:11:07.328186 8930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:11:07.329221 8930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [term 2 LEADER]: Becoming Leader. State: Replica: 33ab827f5f434dd782bfc4e67c989e17, State: Running, Role: LEADER
I20260808 13:11:07.329891 8930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } }
I20260808 13:11:07.330201 8926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:11:07.332453 8932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ab827f5f434dd782bfc4e67c989e17. Latest consensus state: current_term: 2 leader_uuid: "33ab827f5f434dd782bfc4e67c989e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } } }
I20260808 13:11:07.332870 8932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:07.333609 8931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "33ab827f5f434dd782bfc4e67c989e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ab827f5f434dd782bfc4e67c989e17" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38483 } } }
I20260808 13:11:07.334096 8931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:07.334355 8937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:11:07.347807 8937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:11:07.349709 8937 catalog_manager.cc:1259] Loaded cluster ID: 208274c6fb5f4652bdb114c01d484dff
I20260808 13:11:07.349895 8937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:11:07.353960 8937 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:11:07.355456 8937 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33ab827f5f434dd782bfc4e67c989e17: Loaded TSK: 0
I20260808 13:11:07.357033 8937 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:11:07.625197 8928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:07.625531 8928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:07.625667 8928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:07.633100 8928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:07.633216 8928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:07.633365 8928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:11:07.643203 8928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.129.keytab
--principal=oryx/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:38829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=39269
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:07.644640 8928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:07.646013 8928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:07.660369 8952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:07.660581 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:07.660872 8953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:07.661652 8928 server_base.cc:1034] running on GCE node
I20260808 13:11:07.662281 8928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:07.663233 8928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:07.664440 8928 hybrid_clock.cc:648] HybridClock initialized: now 1786194667664377 us; error 57 us; skew 500 ppm
I20260808 13:11:07.664741 8928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:07 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194667, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:07.668031 8928 init.cc:377] Logged in from keytab as oryx/127.31.59.129@KRBTEST.COM (short username oryx)
I20260808 13:11:07.670662 8928 webserver.cc:460] Webserver started at http://127.31.59.129:39269/ using document root <none> and password file <none>
I20260808 13:11:07.671355 8928 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:07.671494 8928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:07.679832 8928 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.004s
I20260808 13:11:07.682540 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:07.683945 8928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:11:07.684103 8928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "80bfdb024976427bb90cc486ba0b0f49"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "ae0b465080e564cfb71d0de1a3bedb3e"
server_key_iv: "265a1c893e39141ff9ba5c42fd6a38bf"
server_key_version: "encryptionkey@0"
I20260808 13:11:07.685111 8928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:07.703524 8928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:07.707557 8928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:07.707820 8928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:07.708811 8928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:07.710191 8928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:07.710305 8928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:07.710414 8928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:07.710489 8928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:07.741925 8928 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:38829
I20260808 13:11:07.741998 9080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:38829 every 8 connection(s)
I20260808 13:11:07.743367 8928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:11:07.746495 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 8928
I20260808 13:11:07.747306 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:34933
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:38483
--webserver_port=37153
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:11:07 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:07 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:11:08.088881 9085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:08.089154 9085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:08.089372 9085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:08.096939 9085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:08.097065 9085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:08.097235 9085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:11:08.107169 9085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.130.keytab
--principal=oryx/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:34933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=37153
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:08.108538 9085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:08.109896 9085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:08.123448 9091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:08.123776 9092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:08.125795 9095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:08.127435 9085 server_base.cc:1034] running on GCE node
I20260808 13:11:08.128392 9085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:08.129910 9085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:08.131201 9085 hybrid_clock.cc:648] HybridClock initialized: now 1786194668131108 us; error 56 us; skew 500 ppm
I20260808 13:11:08.131676 9085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194668, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:08.136302 9085 init.cc:377] Logged in from keytab as oryx/127.31.59.130@KRBTEST.COM (short username oryx)
I20260808 13:11:08.138466 9085 webserver.cc:460] Webserver started at http://127.31.59.130:37153/ using document root <none> and password file <none>
I20260808 13:11:08.139604 9085 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:08.139789 9085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:08.148600 9085 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 13:11:08.152969 9102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:08.154973 9085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:11:08.155202 9085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "e7e8560035e34253af9ff6aedae32abc"
format_stamp: "Formatted at 2026-08-08 13:11:05 on dist-test-slave-49h3"
server_key: "69884fbd7a5bcfda16be17e5d67964f8"
server_key_iv: "608b3f3e46e5a8fcfebce4d951908a61"
server_key_version: "encryptionkey@0"
I20260808 13:11:08.156868 9085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:08.196390 9085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:08.202833 9085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:08.203217 9085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:08.204797 9085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:08.207041 9085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:08.207188 9085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:08.207332 9085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:08.207417 9085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:08.243497 9085 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34933
I20260808 13:11:08.243507 9221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34933 every 8 connection(s)
I20260808 13:11:08.246063 9085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:11:08.250686 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9085
I20260808 13:11:08.251447 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:33379
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:38483
--webserver_port=41037
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:11:08.790807 9226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:08.791252 9226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:08.791474 9226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:08.803419 9226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:08.803574 9226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:08.803817 9226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:11:08.819970 9226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/krb5kdc/oryx_127.31.59.131.keytab
--principal=oryx/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=41037
--tserver_master_addrs=127.31.59.190:38483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:08.822064 9226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:08.823915 9226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:08.842459 9233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:08.846704 9232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:08.847015 9226 server_base.cc:1034] running on GCE node
W20260808 13:11:08.847519 9235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:08.848482 9226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:08.849973 9226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:08.851292 9226 hybrid_clock.cc:648] HybridClock initialized: now 1786194668851233 us; error 45 us; skew 500 ppm
I20260808 13:11:08.851742 9226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194668, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:08.856549 9226 init.cc:377] Logged in from keytab as oryx/127.31.59.131@KRBTEST.COM (short username oryx)
I20260808 13:11:08.859053 9226 webserver.cc:460] Webserver started at http://127.31.59.131:41037/ using document root <none> and password file <none>
I20260808 13:11:08.859999 9226 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:08.860162 9226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:08.868094 9226 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:11:08.871794 9242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:08.874850 9226 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 13:11:08.875046 9226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "e6e79c2222e547cd8c4abb7bbdb9b611"
format_stamp: "Formatted at 2026-08-08 13:11:06 on dist-test-slave-49h3"
server_key: "a33a23ce00dc9d659bfdc74c904e6104"
server_key_iv: "19a2204410d53a70fb556d481e6b966d"
server_key_version: "encryptionkey@0"
I20260808 13:11:08.876487 9226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260808 13:11:08.921842 9084 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.162s user 0.003s sys 0.013s
I20260808 13:11:08.922451 9226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260808 13:11:08.924292 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:07.746349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:46077 (local address 127.31.59.190:38483)
0808 13:11:07.747043 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:07.747055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:07.750275 (+ 3220us) server_negotiation.cc:407] Connection header received
0808 13:11:07.752445 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:07.752459 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:07.752829 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:07.753160 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:07.754763 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:07.756016 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:07.757578 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:07.758461 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:07.758646 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.923334 (+1164688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:46077: BlockingRecv error: recv got EOF from 127.31.59.129:46077 (error 108)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":269,"threads_started":1}
I20260808 13:11:08.925330 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:07.746587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:46077)
0808 13:11:07.750078 (+ 3491us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:07.750118 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:07.751893 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:07.753356 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:07.753369 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:07.754014 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:07.754563 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:07.754580 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:07.756191 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:07.756195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:07.757422 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:07.757434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:07.758693 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:07.758698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:07.758918 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:07.759552 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:07.759574 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:08.922919 (+1163345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3221,"thread_start_us":3044,"threads_started":1}
W20260808 13:11:08.926376 9081 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:38483 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.190:38483: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
I20260808 13:11:08.927999 9226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:08.928344 9226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:08.929502 9226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:08.931228 9226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:08.931337 9226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:08.931452 9226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:08.931524 9226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:08.939293 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:08.927185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:43505)
0808 13:11:08.927608 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:08.927629 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:08.927771 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:08.928154 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:08.928159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:08.928680 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:08.928927 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.928940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.930407 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.930415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.932127 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.932145 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.933141 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.933148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.933362 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.934064 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:08.934090 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:08.938918 (+ 4828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":110}
W20260808 13:11:08.939476 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:08.927286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43505 (local address 127.31.59.190:38483)
0808 13:11:08.927471 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:08.927476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:08.927716 (+ 240us) server_negotiation.cc:407] Connection header received
0808 13:11:08.927900 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:08.927905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:08.927993 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:08.928116 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:08.929105 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.930229 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.932292 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.932910 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.933064 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.939205 (+ 6141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:43505: BlockingRecv error: recv got EOF from 127.31.59.129:43505 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:08 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:08.952369 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:08.940487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:45881)
0808 13:11:08.940760 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:08.940781 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:08.940896 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:08.941301 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:08.941306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:08.941784 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:08.942020 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.942032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.943546 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.943552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.945199 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.945216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.946303 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.946311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.946524 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.947289 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:08.947314 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:08.952007 (+ 4693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152}
W20260808 13:11:08.952497 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:08.940675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:45881 (local address 127.31.59.190:38483)
0808 13:11:08.940854 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:08.940859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:08.940881 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:11:08.941009 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:08.941015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:08.941108 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:08.941238 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:08.942183 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.943390 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.945396 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.946100 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.946238 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.952234 (+ 5996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:45881: BlockingRecv error: recv got EOF from 127.31.59.129:45881 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 13:11:08.953523 9081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:11:08.984171 9226 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:33379
I20260808 13:11:08.984221 9360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:33379 every 8 connection(s)
I20260808 13:11:08.985709 9226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:11:08.992894 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9226
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:09.023249 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:08.994061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.31.59.190:38483)
0808 13:11:09.005909 (+ 11848us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:09.005915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:09.006462 (+ 547us) server_negotiation.cc:407] Connection header received
0808 13:11:09.006660 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:09.006667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:09.006756 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:09.006883 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:09.009116 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.010375 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.012443 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.012979 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.013116 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.016786 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:09.016797 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:09.016821 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:09.016868 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:09.019880 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:09.020545 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:09.020553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:09.020561 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:09.020737 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:09.021120 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:09.021124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:09.021127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:09.021679 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:09.021992 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:09.022430 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 13:11:09.022939 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11712,"thread_start_us":11524,"threads_started":1}
W20260808 13:11:09.377954 9225 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.115s user 0.000s sys 0.019s
W20260808 13:11:09.379311 9224 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:08.249367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:34511 (local address 127.31.59.190:38483)
0808 13:11:08.249844 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:08.249848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:08.254160 (+ 4312us) server_negotiation.cc:407] Connection header received
0808 13:11:08.254252 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:08.254256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:08.254337 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:08.254453 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:08.257536 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.258850 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.261112 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.261863 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.262003 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.378947 (+1116944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:34511: BlockingRecv error: recv got EOF from 127.31.59.130:34511 (error 108)
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260808 13:11:09.379870 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:08.250335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:34511)
0808 13:11:08.251323 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:08.251377 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:08.253780 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:08.254838 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:08.254853 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:08.256258 (+ 1405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:08.257223 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.257253 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.259069 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.259074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.260937 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.260955 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.262110 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.262116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.262462 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:08.263273 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:08.263307 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:09.378765 (+1115458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":386,"threads_started":1}
W20260808 13:11:09.380761 9222 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:38483 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.190:38483: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:09.390657 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:09.381508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:51867)
0808 13:11:09.381822 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:09.381844 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:09.381987 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:09.382347 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:09.382352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:09.382725 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:09.382926 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.382935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.383952 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.383957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.384898 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.384909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.385600 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.385606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.385767 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.386588 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:09.386610 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:09.390455 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174}
W20260808 13:11:09.391023 9224 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:09.381501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:51867 (local address 127.31.59.190:38483)
0808 13:11:09.381644 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:09.381648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:09.381911 (+ 263us) server_negotiation.cc:407] Connection header received
0808 13:11:09.382106 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:09.382109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:09.382197 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:09.382294 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:09.383055 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.383813 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.385028 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.385425 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.385554 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.390754 (+ 5200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:51867: BlockingRecv error: recv got EOF from 127.31.59.130:51867 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:09.400631 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:09.391773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:53371)
0808 13:11:09.391948 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:09.391966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:09.392054 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:09.392392 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:09.392395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:09.392647 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:09.392801 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.392808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.394262 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.394265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.395175 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.395186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.396102 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.396106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.396224 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.396826 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:09.396842 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:09.400445 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 13:11:09.401000 9222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:11:09.401521 9224 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:09.391921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:53371 (local address 127.31.59.190:38483)
0808 13:11:09.392090 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:09.392096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:09.392117 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:11:09.392170 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:09.392176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:09.392272 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:09.392379 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:09.392913 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.394145 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.395288 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.395981 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.396105 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.401178 (+ 5073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:53371: BlockingRecv error: recv got EOF from 127.31.59.130:53371 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:09 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:09.963735 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:09.954132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:53689)
0808 13:11:09.954820 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:09.954836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:09.954967 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:09.955304 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:09.955309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:09.955800 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:09.955954 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.955963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.956951 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.956955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.957959 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:09.957972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.958681 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.958689 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:09.958857 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.959494 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:09.959510 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:09.963409 (+ 3899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":441,"threads_started":1}
W20260808 13:11:09.963999 9367 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:09.954149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:53689 (local address 127.31.59.190:38483)
0808 13:11:09.954621 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:09.954627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:09.954914 (+ 287us) server_negotiation.cc:407] Connection header received
0808 13:11:09.955077 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:09.955082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:09.955165 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:09.955272 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:09.956090 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.956819 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.958089 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:09.958509 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:09.958616 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.963711 (+ 5095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:53689: BlockingRecv error: recv got EOF from 127.31.59.129:53689 (error 108)
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
W20260808 13:11:10.063935 9363 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.063s user 0.008s sys 0.008s
W20260808 13:11:10.065143 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:08.988460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:60035 (local address 127.31.59.190:38483)
0808 13:11:08.988635 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:08.988640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:08.990477 (+ 1837us) server_negotiation.cc:407] Connection header received
0808 13:11:08.992340 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:08.992346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:08.992429 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:08.992521 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:08.994409 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.995621 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.998282 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.999007 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.999106 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.064896 (+1065790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:60035: BlockingRecv error: recv got EOF from 127.31.59.131:60035 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:11:10.065812 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:08.989426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:60035)
0808 13:11:08.990297 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:08.990346 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:08.992064 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:08.992777 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:08.992786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:08.993352 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:08.994171 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.994199 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.995890 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.995896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.998083 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:08.998102 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:08.999503 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:08.999510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:08.999847 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:09.000757 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:09.000792 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:10.064742 (+1063950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":341,"threads_started":1}
W20260808 13:11:10.066735 9361 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:38483 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.190:38483: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:10.075531 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:10.067416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:39243)
0808 13:11:10.067594 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:10.067610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:10.067703 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:10.068030 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:10.068034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:10.068380 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:10.068543 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.068551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.069639 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.069642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.070585 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.070596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.071230 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.071234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.071345 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.071861 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:10.071878 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:10.075320 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 13:11:10.075739 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:10.067485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39243 (local address 127.31.59.190:38483)
0808 13:11:10.067703 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:10.067707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:10.067725 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:11:10.067808 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:10.067811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:10.067871 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:10.067957 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:10.068698 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.069510 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.070688 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.071095 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.071193 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.075509 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:39243: BlockingRecv error: recv got EOF from 127.31.59.131:39243 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:10.083134 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:10.076156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:48163)
0808 13:11:10.076290 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:10.076308 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:10.076387 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:10.076620 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:10.076623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:10.076908 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:10.077050 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.077057 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.077974 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.077977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.078922 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.078933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.079519 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.079523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.079655 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.080118 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:10.080133 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:10.082944 (+ 2811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 13:11:10.083321 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:10.076199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:48163 (local address 127.31.59.190:38483)
0808 13:11:10.076337 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:10.076341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:10.076356 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:10.076464 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:10.076467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:10.076520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:10.076590 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:10.077146 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.077886 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.079025 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.079422 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.079502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.083117 (+ 3615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:48163: BlockingRecv error: recv got EOF from 127.31.59.131:48163 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 13:11:10.083436 9361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:10.410666 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:10.401613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:37495)
0808 13:11:10.402251 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:10.402268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:10.402387 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:10.402678 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:10.402682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:10.403103 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:10.403269 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.403278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.404216 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.404219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.405169 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.405180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.406051 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.406059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.406199 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.406844 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:10.406867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:10.410426 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":380,"threads_started":1}
W20260808 13:11:10.410888 9083 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:10.401707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:37495 (local address 127.31.59.190:38483)
0808 13:11:10.401916 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:10.401921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:10.402336 (+ 415us) server_negotiation.cc:407] Connection header received
0808 13:11:10.402491 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:10.402495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:10.402574 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:10.402651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:10.403380 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.404110 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.405286 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.405829 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.405931 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.410639 (+ 4708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:37495: BlockingRecv error: recv got EOF from 127.31.59.130:37495 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:10 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:10.973351 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:10.964756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:34103)
0808 13:11:10.965292 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:10.965310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:10.965416 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:10.965715 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:10.965718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:10.966151 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:10.966350 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.966360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.967391 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.967394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.968342 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:10.968354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.968999 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.969003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:10.969123 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.969712 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:10.969736 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:10.973059 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":270,"threads_started":1}
W20260808 13:11:10.973592 9371 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:10.964817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:34103 (local address 127.31.59.190:38483)
0808 13:11:10.965334 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:10.965338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:10.965372 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:11:10.965534 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:10.965538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:10.965598 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:10.965680 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:10.966493 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.967280 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.968448 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:10.968883 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:10.968973 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:10.973292 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:34103: BlockingRecv error: recv got EOF from 127.31.59.129:34103 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":294,"threads_started":1}
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:11.097489 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:11.083940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:58809)
0808 13:11:11.084470 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:11.084485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:11.084616 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:11.084931 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:11.084936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:11.085406 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:11.085581 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.085591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.086593 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.086597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.087563 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.087575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.088324 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.088330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.088482 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.093972 (+ 5490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:11.093992 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:11.097216 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":306,"threads_started":1}
W20260808 13:11:11.097666 9371 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:11.084035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:58809 (local address 127.31.59.190:38483)
0808 13:11:11.084221 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:11.084226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:11.084567 (+ 341us) server_negotiation.cc:407] Connection header received
0808 13:11:11.084727 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:11.084731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:11.084805 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:11.084886 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:11.085717 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.086467 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.087694 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.088142 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.088250 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.097416 (+ 9166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:58809: BlockingRecv error: recv got EOF from 127.31.59.131:58809 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:11.420207 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:11.411600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:55147)
0808 13:11:11.412123 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:11.412141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:11.412271 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:11.412620 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:11.412622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:11.413056 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:11.413289 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.413306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.414568 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.414574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.415515 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.415527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.416153 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.416157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.416284 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.416840 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:11.416858 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:11.419964 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":288,"threads_started":1}
W20260808 13:11:11.420396 9371 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:11.411716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:55147 (local address 127.31.59.190:38483)
0808 13:11:11.411886 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:11.411890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:11.412212 (+ 322us) server_negotiation.cc:407] Connection header received
0808 13:11:11.412423 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:11.412428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:11.412499 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:11.412585 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:11.413478 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.414415 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.415632 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.416036 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.416125 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.420150 (+ 4025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:55147: BlockingRecv error: recv got EOF from 127.31.59.130:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:11 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:11.982820 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:11.974324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:57995)
0808 13:11:11.974817 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:11.974835 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:11.974945 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:11.975387 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:11.975390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:11.975743 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:11.975923 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.975932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.976875 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.976879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.977843 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:11.977855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.978460 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.978464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:11.978579 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.979124 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:11.979140 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:11.982487 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
W20260808 13:11:11.983019 9375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:11.974492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:57995 (local address 127.31.59.190:38483)
0808 13:11:11.975123 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:11.975127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:11.975142 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:11.975188 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:11.975191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:11.975271 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:11.975357 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:11.976038 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.976766 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.977944 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:11.978354 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:11.978436 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:11.982707 (+ 4271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:57995: BlockingRecv error: recv got EOF from 127.31.59.129:57995 (error 108)
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":405,"threads_started":1}
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:12.107347 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:12.098472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:46675)
0808 13:11:12.098997 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:12.099012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:12.099146 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:12.099474 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:12.099480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:12.099891 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:12.100086 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.100096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.101144 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.101148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.102363 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.102378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.103033 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.103037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.103162 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.103709 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:12.103726 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:12.107079 (+ 3353us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":285,"threads_started":1}
W20260808 13:11:12.107465 9375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:12.098584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:46675 (local address 127.31.59.190:38483)
0808 13:11:12.098814 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:12.098821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:12.099094 (+ 273us) server_negotiation.cc:407] Connection header received
0808 13:11:12.099261 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:12.099266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:12.099333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:12.099417 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:12.100230 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.100987 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.102487 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.102915 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.103010 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.107267 (+ 4257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:46675: BlockingRecv error: recv got EOF from 127.31.59.131:46675 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:12.430079 9377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:12.421137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:57109)
0808 13:11:12.421626 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:12.421643 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:12.421781 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:12.422098 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:12.422103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:12.422504 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:12.422670 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.422680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.423664 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.423669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.424690 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.424704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.425526 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.425531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.425706 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.426241 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:12.426259 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:12.429851 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":231,"threads_started":1}
W20260808 13:11:12.430251 9375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:12.421182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:57109 (local address 127.31.59.190:38483)
0808 13:11:12.421374 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:12.421380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:12.421720 (+ 340us) server_negotiation.cc:407] Connection header received
0808 13:11:12.421900 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:12.421906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:12.421973 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:12.422051 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:12.422820 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.423538 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.424830 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.425363 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.425498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.430024 (+ 4526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:57109: BlockingRecv error: recv got EOF from 127.31.59.130:57109 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:12 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:12.992980 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:12.983918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:48423)
0808 13:11:12.984765 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:12.984780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:12.984906 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:12.985262 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:12.985267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:12.985712 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:12.985889 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.985900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.986888 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.986893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.987887 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:12.987899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.988563 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.988569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:12.988746 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.989373 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:12.989390 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:12.992628 (+ 3238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":628,"threads_started":1}
W20260808 13:11:12.993141 9379 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:12.983991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:48423 (local address 127.31.59.190:38483)
0808 13:11:12.984443 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:12.984447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:12.984868 (+ 421us) server_negotiation.cc:407] Connection header received
0808 13:11:12.985020 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:12.985024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:12.985108 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:12.985197 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:12.986044 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.986762 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.988024 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:12.988423 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:12.988532 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:12.992872 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:48423: BlockingRecv error: recv got EOF from 127.31.59.129:48423 (error 108)
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":245,"threads_started":1}
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:13.117030 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:13.108271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:45195)
0808 13:11:13.108791 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:13.108808 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:13.108939 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:13.109246 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:13.109252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:13.109686 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:13.109876 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.109887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.110885 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.110891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.111958 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.111973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.112783 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.112789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.112971 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:13.113562 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:13.113579 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:13.116774 (+ 3195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":263,"threads_started":1}
W20260808 13:11:13.117121 9379 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:13.108374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:45195 (local address 127.31.59.190:38483)
0808 13:11:13.108633 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:13.108640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:13.108881 (+ 241us) server_negotiation.cc:407] Connection header received
0808 13:11:13.109050 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:13.109055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:13.109114 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:13.109197 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:13.110020 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.110747 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.112123 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.112617 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.112763 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:13.116945 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:45195: BlockingRecv error: recv got EOF from 127.31.59.131:45195 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:13.439563 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:13.431025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:37045)
0808 13:11:13.431527 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:13.431543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:13.431668 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:13.432023 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:13.432028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:13.432435 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:13.432598 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.432608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.433677 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.433683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.434654 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.434666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.435347 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.435352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.435516 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:13.436032 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:13.436048 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:13.439225 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":267,"threads_started":1}
W20260808 13:11:13.439667 9379 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:13.431094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:37045 (local address 127.31.59.190:38483)
0808 13:11:13.431285 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:13.431290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:13.431619 (+ 329us) server_negotiation.cc:407] Connection header received
0808 13:11:13.431796 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:13.431801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:13.431881 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:13.431966 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:13.432743 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.433559 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.434785 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.435206 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.435321 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:13.439445 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:37045: BlockingRecv error: recv got EOF from 127.31.59.130:37045 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:13 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:14.002568 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:13.993949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:33827)
0808 13:11:13.994503 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:13.994520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:13.994638 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:13.994950 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:13.994953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:13.995402 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:13.995565 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.995574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.996642 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.996647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.997622 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:13.997635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.998325 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.998331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:13.998489 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:13.999050 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:13.999068 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:14.002256 (+ 3188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":268,"threads_started":1}
W20260808 13:11:14.002743 9383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:13.994040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:33827 (local address 127.31.59.190:38483)
0808 13:11:13.994510 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:13.994515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:13.994591 (+ 76us) server_negotiation.cc:407] Connection header received
0808 13:11:13.994744 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:13.994748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:13.994825 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:13.994918 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:13.995721 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.996508 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.997770 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:13.998174 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:13.998281 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:14.002467 (+ 4186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:33827: BlockingRecv error: recv got EOF from 127.31.59.129:33827 (error 108)
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":256,"threads_started":1}
Aug 08 13:11:14 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:14 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:14.127086 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:14.118142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:39449)
0808 13:11:14.118662 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:14.118677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:14.118809 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:14.119129 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:14.119134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:14.119550 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:14.119725 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:14.119734 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.120755 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.120759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:14.121782 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:14.121793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.122437 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.122443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:14.122622 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:14.123185 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:14.123202 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:14.126727 (+ 3525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":288,"threads_started":1}
W20260808 13:11:14.127182 9383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:14.118208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39449 (local address 127.31.59.190:38483)
0808 13:11:14.118409 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:14.118415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:14.118785 (+ 370us) server_negotiation.cc:407] Connection header received
0808 13:11:14.118925 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:14.118929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:14.118991 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:14.119071 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:14.119874 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.120610 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.121918 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.122302 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.122410 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:14.126939 (+ 4529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:39449: BlockingRecv error: recv got EOF from 127.31.59.131:39449 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 13:11:14 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:14 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:14.449486 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:14.440500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:54885)
0808 13:11:14.441017 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:14.441033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:14.441164 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:14.441518 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:14.441522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:14.441973 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:14.442131 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:14.442141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.443130 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.443135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:14.444169 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:14.444184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.445131 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.445137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:14.445313 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:14.445933 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:14.445950 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:14.449221 (+ 3271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":260,"threads_started":1}
W20260808 13:11:14.449673 9383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:14.440706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54885 (local address 127.31.59.190:38483)
0808 13:11:14.440913 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:14.440920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:14.441107 (+ 187us) server_negotiation.cc:407] Connection header received
0808 13:11:14.441287 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:14.441290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:14.441363 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:14.441466 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:14.442284 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.442990 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.444311 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:14.444966 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:14.445092 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:14.449409 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:54885: BlockingRecv error: recv got EOF from 127.31.59.130:54885 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:15.012168 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:15.003475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:50509)
0808 13:11:15.003947 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:15.003963 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:15.004074 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:15.004519 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:15.004522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:15.004945 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:15.005127 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.005136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.006168 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.006172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.007082 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.007094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.007759 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.007763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.007882 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.008418 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:15.008434 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:15.011854 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":249,"threads_started":1}
W20260808 13:11:15.012287 9387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:15.003580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:50509 (local address 127.31.59.190:38483)
0808 13:11:15.004270 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:15.004275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:15.004291 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:11:15.004336 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:15.004339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:15.004405 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:15.004491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:15.005258 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.006021 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.007209 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.007633 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.007723 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.012025 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:50509: BlockingRecv error: recv got EOF from 127.31.59.129:50509 (error 108)
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":481,"threads_started":1}
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:15.137094 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:15.128221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:42543)
0808 13:11:15.128810 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:15.128830 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:15.128954 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:15.129340 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:15.129345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:15.129939 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:15.130136 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.130149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.131215 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.131218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.132227 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.132238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.132914 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.132918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.133043 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.133630 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:15.133647 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:15.136865 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
W20260808 13:11:15.137226 9387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:15.128360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:42543 (local address 127.31.59.190:38483)
0808 13:11:15.128582 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:15.128588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:15.128920 (+ 332us) server_negotiation.cc:407] Connection header received
0808 13:11:15.129083 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:15.129088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:15.129173 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:15.129261 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:15.130298 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.131097 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.132371 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.132805 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.132895 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.137049 (+ 4154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:42543: BlockingRecv error: recv got EOF from 127.31.59.131:42543 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:15 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:15.459079 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:15.450404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:51565)
0808 13:11:15.450840 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:15.450855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:15.450989 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:15.451325 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:15.451330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:15.451754 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:15.451921 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.451931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.452952 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.452955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.453913 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:15.453925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.454618 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.454624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:15.454794 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.455341 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:15.455359 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:15.458812 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
W20260808 13:11:15.459316 9387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:15.450455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:51565 (local address 127.31.59.190:38483)
0808 13:11:15.450643 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:15.450649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:15.450942 (+ 293us) server_negotiation.cc:407] Connection header received
0808 13:11:15.451105 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:15.451110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:15.451182 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:15.451263 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:15.452089 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.452815 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.454056 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:15.454456 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:15.454571 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:15.459017 (+ 4446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:51565: BlockingRecv error: recv got EOF from 127.31.59.130:51565 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:16.022334 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:16.013161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:36283)
0808 13:11:16.014009 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:16.014025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:16.014144 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:16.014457 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:16.014460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:16.014866 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:16.015039 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.015047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.016017 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.016020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.017139 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.017156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.018036 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.018043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.018210 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.018833 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:16.018852 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:16.022074 (+ 3222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":549,"threads_started":1}
W20260808 13:11:16.022539 9391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:16.013238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:36283 (local address 127.31.59.190:38483)
0808 13:11:16.013812 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:16.013819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:16.014106 (+ 287us) server_negotiation.cc:407] Connection header received
0808 13:11:16.014267 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:16.014271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:16.014338 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:16.014425 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:16.015180 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.015914 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.017279 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.017884 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.018007 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.022300 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:36283: BlockingRecv error: recv got EOF from 127.31.59.129:36283 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":286,"threads_started":1}
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:16.151988 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:16.138075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:54371)
0808 13:11:16.138568 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:16.138584 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:16.138701 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:16.138966 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:16.138969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:16.139404 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:16.139578 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.139587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.140534 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.140538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.141489 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.141503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.142101 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.142105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.142249 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.148197 (+ 5948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:16.148217 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:16.151757 (+ 3540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
W20260808 13:11:16.152159 9391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:16.138106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:54371 (local address 127.31.59.190:38483)
0808 13:11:16.138272 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:16.138277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:16.138649 (+ 372us) server_negotiation.cc:407] Connection header received
0808 13:11:16.138794 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:16.138797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:16.138860 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:16.138936 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:16.139683 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.140431 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.141593 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.141983 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.142070 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.151965 (+ 9895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:54371: BlockingRecv error: recv got EOF from 127.31.59.131:54371 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:16 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:16.468787 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:16.460240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:43481)
0808 13:11:16.460816 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:16.460833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:16.460945 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:16.461213 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:16.461216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:16.461643 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:16.461841 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.461850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.462785 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.462790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.463821 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:16.463834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.464461 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.464466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:16.464576 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.465102 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:16.465119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:16.468529 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":256,"threads_started":1}
W20260808 13:11:16.468961 9391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:16.460423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:43481 (local address 127.31.59.190:38483)
0808 13:11:16.460630 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:16.460635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:16.460894 (+ 259us) server_negotiation.cc:407] Connection header received
0808 13:11:16.461037 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:16.461040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:16.461114 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:16.461187 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:16.461948 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.462663 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.463933 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:16.464337 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:16.464449 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:16.468731 (+ 4282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:43481: BlockingRecv error: recv got EOF from 127.31.59.130:43481 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:17.032254 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:17.023216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:56479)
0808 13:11:17.023726 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:17.023757 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:17.023864 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:17.024274 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:17.024277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:17.024694 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:17.024905 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.024916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.025935 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.025938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.026867 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.026878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.027545 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.027551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.027744 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.028454 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:17.028479 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:17.031997 (+ 3518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":246,"threads_started":1}
W20260808 13:11:17.032426 9396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:17.023296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:56479 (local address 127.31.59.190:38483)
0808 13:11:17.024030 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:17.024034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:17.024049 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:17.024091 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:17.024094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:17.024150 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:17.024230 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:17.025035 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.025828 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.026966 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.027377 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.027491 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.032201 (+ 4710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:56479: BlockingRecv error: recv got EOF from 127.31.59.129:56479 (error 108)
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":550,"threads_started":1}
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:17.162047 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:17.152876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:37699)
0808 13:11:17.153517 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:17.153533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:17.153645 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:17.153912 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:17.153915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:17.154421 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:17.154639 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.154653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.155821 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.155826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.156763 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.156776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.157523 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.157530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.157685 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.158340 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:17.158363 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:17.161729 (+ 3366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":298,"threads_started":1}
W20260808 13:11:17.162283 9396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:17.153028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:37699 (local address 127.31.59.190:38483)
0808 13:11:17.153205 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:17.153209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:17.153596 (+ 387us) server_negotiation.cc:407] Connection header received
0808 13:11:17.153734 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:17.153738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:17.153800 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:17.153878 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:17.154804 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.155677 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.156902 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.157318 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.157423 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.161941 (+ 4518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:37699: BlockingRecv error: recv got EOF from 127.31.59.131:37699 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:17 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:17.478583 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:17.469699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:50093)
0808 13:11:17.470195 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:17.470211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:17.470336 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:17.470610 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:17.470613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:17.471007 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:17.471181 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.471190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.472296 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.472300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.473224 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:17.473244 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.473865 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.473870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:17.473992 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.474612 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:17.474629 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:17.478256 (+ 3627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
W20260808 13:11:17.478729 9396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:17.469805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:50093 (local address 127.31.59.190:38483)
0808 13:11:17.470030 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:17.470036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:17.470296 (+ 260us) server_negotiation.cc:407] Connection header received
0808 13:11:17.470423 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:17.470427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:17.470503 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:17.470584 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:17.471311 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.472183 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.473334 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:17.473729 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:17.473814 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:17.478493 (+ 4679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:50093: BlockingRecv error: recv got EOF from 127.31.59.130:50093 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:18.042086 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:18.033174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:55781)
0808 13:11:18.033763 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:18.033783 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:18.033920 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:18.034233 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:18.034237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:18.034643 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:18.034821 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.034830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.036057 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.036061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.037214 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.037228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.037871 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.037875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.038000 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.038526 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:18.038542 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:18.041774 (+ 3232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":308,"threads_started":1}
W20260808 13:11:18.042232 9400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:18.033231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55781 (local address 127.31.59.190:38483)
0808 13:11:18.033725 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:18.033733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:18.033860 (+ 127us) server_negotiation.cc:407] Connection header received
0808 13:11:18.034023 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:18.034027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:18.034090 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:18.034180 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:18.034941 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.035928 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.037336 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.037759 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.037850 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.041980 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:55781: BlockingRecv error: recv got EOF from 127.31.59.129:55781 (error 108)
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:18.171653 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:18.163039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:44131)
0808 13:11:18.163538 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:18.163554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:18.163671 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:18.163981 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:18.163985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:18.164451 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:18.164639 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.164650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.165671 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.165675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.166884 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.166903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.167594 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.167599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.167742 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.168327 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:18.168346 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:18.171400 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":256,"threads_started":1}
W20260808 13:11:18.171804 9400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:18.163102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:44131 (local address 127.31.59.190:38483)
0808 13:11:18.163312 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:18.163319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:18.163628 (+ 309us) server_negotiation.cc:407] Connection header received
0808 13:11:18.163777 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:18.163780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:18.163843 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:18.163921 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:18.164774 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.165577 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.167019 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.167426 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.167534 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.171580 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:44131: BlockingRecv error: recv got EOF from 127.31.59.131:44131 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:18 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:18.488420 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:18.479578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:54239)
0808 13:11:18.480128 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:18.480143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:18.480256 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:18.480536 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:18.480540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:18.480961 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:18.481112 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.481121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.482188 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.482193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.483174 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:18.483187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.483840 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.483844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:18.483966 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.484482 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:18.484498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:18.488108 (+ 3610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":307,"threads_started":1}
W20260808 13:11:18.488543 9400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:18.479635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54239 (local address 127.31.59.190:38483)
0808 13:11:18.479791 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:18.479795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:18.480208 (+ 413us) server_negotiation.cc:407] Connection header received
0808 13:11:18.480344 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:18.480347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:18.480423 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:18.480503 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:18.481229 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.482054 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.483298 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:18.483710 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:18.483801 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:18.488308 (+ 4507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:54239: BlockingRecv error: recv got EOF from 127.31.59.130:54239 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:19.052368 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:19.043095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:51867)
0808 13:11:19.043579 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:19.043595 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:19.043701 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:19.044121 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:19.044124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:19.044494 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:19.044653 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.044662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.045707 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.045712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.047260 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.047279 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.047975 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.047979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.048138 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.048701 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:19.048717 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:19.052116 (+ 3399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":261,"threads_started":1}
W20260808 13:11:19.052513 9404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:19.043205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:51867 (local address 127.31.59.190:38483)
0808 13:11:19.043867 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:19.043871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:19.043886 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:11:19.043931 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:19.043934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:19.044003 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:19.044088 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:19.044784 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.045518 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.047413 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.047856 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.047949 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.052291 (+ 4342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:51867: BlockingRecv error: recv got EOF from 127.31.59.129:51867 (error 108)
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":455,"threads_started":1}
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:19.181988 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:19.172607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:41847)
0808 13:11:19.173130 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:19.173147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:19.173266 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:19.173549 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:19.173552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:19.173978 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:19.174135 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.174145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.175082 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.175085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.176218 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.176236 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.177372 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.177379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.177600 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.178259 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:19.178284 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:19.181751 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":264,"threads_started":1}
W20260808 13:11:19.182083 9404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:19.172704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:41847 (local address 127.31.59.190:38483)
0808 13:11:19.172871 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:19.172875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:19.173213 (+ 338us) server_negotiation.cc:407] Connection header received
0808 13:11:19.173355 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:19.173359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:19.173420 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:19.173519 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:19.174261 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.174979 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.176391 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.177195 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.177315 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.181892 (+ 4577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:41847: BlockingRecv error: recv got EOF from 127.31.59.131:41847 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:19 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:19.497956 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:19.489520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:51059)
0808 13:11:19.490042 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:19.490057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:19.490168 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:19.490441 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:19.490444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:19.490855 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:19.491016 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.491026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.491961 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.491965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.492947 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:19.492959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.493567 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.493571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:19.493686 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.494314 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:19.494331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:19.497703 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":300,"threads_started":1}
W20260808 13:11:19.498070 9404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:19.489543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:51059 (local address 127.31.59.190:38483)
0808 13:11:19.489784 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:19.489792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:19.490134 (+ 342us) server_negotiation.cc:407] Connection header received
0808 13:11:19.490256 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:19.490260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:19.490336 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:19.490413 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:19.491125 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.491828 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.493052 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:19.493435 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:19.493551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:19.497867 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:51059: BlockingRecv error: recv got EOF from 127.31.59.130:51059 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:20.062417 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:20.053400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:32801)
0808 13:11:20.054333 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:20.054354 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:20.054516 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:20.054939 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:20.054942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:20.055471 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:20.055660 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.055670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.056588 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.056592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.057541 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.057555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.058175 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.058179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.058310 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.058848 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:20.058864 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:20.062187 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":654,"threads_started":1}
W20260808 13:11:20.062678 9408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:20.053575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:32801 (local address 127.31.59.190:38483)
0808 13:11:20.054148 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:20.054154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:20.054444 (+ 290us) server_negotiation.cc:407] Connection header received
0808 13:11:20.054689 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:20.054695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:20.054784 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:20.054879 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:20.055783 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.056493 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.057646 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.058063 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.058147 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.062402 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:32801: BlockingRecv error: recv got EOF from 127.31.59.129:32801 (error 108)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":292,"threads_started":1}
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:20.191946 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:20.182989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:54797)
0808 13:11:20.183601 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:20.183617 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:20.183757 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:20.184135 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:20.184140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:20.184634 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:20.184864 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.184874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.186065 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.186069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.187053 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.187065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.187710 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.187714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.187825 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.188403 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:20.188420 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:20.191675 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":385,"threads_started":1}
W20260808 13:11:20.192036 9408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:20.183005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:54797 (local address 127.31.59.190:38483)
0808 13:11:20.183225 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:20.183328 (+ 103us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:20.183716 (+ 388us) server_negotiation.cc:407] Connection header received
0808 13:11:20.183904 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:20.183909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:20.183990 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:20.184109 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:20.185012 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.185919 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.187189 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.187591 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.187686 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.191846 (+ 4160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:54797: BlockingRecv error: recv got EOF from 127.31.59.131:54797 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:20 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:20.507714 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:20.498787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:49523)
0808 13:11:20.499380 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:20.499396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:20.499512 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:20.499808 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:20.499811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:20.500226 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:20.500405 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.500415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.501374 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.501377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.502413 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:20.502428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.503343 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.503349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:20.503532 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.504155 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:20.504172 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:20.507493 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":362,"threads_started":1}
W20260808 13:11:20.508035 9408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:20.498920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:49523 (local address 127.31.59.190:38483)
0808 13:11:20.499128 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:20.499133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:20.499465 (+ 332us) server_negotiation.cc:407] Connection header received
0808 13:11:20.499601 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:20.499605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:20.499683 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:20.499770 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:20.500525 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.501273 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.502573 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:20.503184 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:20.503307 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:20.507666 (+ 4359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:49523: BlockingRecv error: recv got EOF from 127.31.59.130:49523 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:21.071771 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:21.063358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:58663)
0808 13:11:21.063913 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:21.063930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:21.064057 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:21.064358 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:21.064361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:21.064792 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:21.064972 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.064981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.065959 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.065962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.066886 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.066897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.067554 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.067559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.067686 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.068244 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:21.068267 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:21.071505 (+ 3238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":293,"threads_started":1}
W20260808 13:11:21.071967 9411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:21.063387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:58663 (local address 127.31.59.190:38483)
0808 13:11:21.063879 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:21.063884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:21.064008 (+ 124us) server_negotiation.cc:407] Connection header received
0808 13:11:21.064155 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:21.064159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:21.064221 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:21.064307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:21.065094 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.065857 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.067005 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.067412 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.067503 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.071706 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:58663: BlockingRecv error: recv got EOF from 127.31.59.129:58663 (error 108)
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":247,"threads_started":1}
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:21.204453 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:21.192843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:33665)
0808 13:11:21.193326 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:21.193342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:21.193497 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:21.193814 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:21.193819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:21.194265 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:21.194461 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.194471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.195453 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.195458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.196376 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.196388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.197070 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.197076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.197248 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.200559 (+ 3311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:21.200577 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:21.204136 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":254,"threads_started":1}
W20260808 13:11:21.204749 9411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:21.192919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:33665 (local address 127.31.59.190:38483)
0808 13:11:21.193098 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:21.193104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:21.193420 (+ 316us) server_negotiation.cc:407] Connection header received
0808 13:11:21.193609 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:21.193614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:21.193679 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:21.193781 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:21.194600 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.195327 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.196508 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.196923 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.197029 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.204367 (+ 7338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:33665: BlockingRecv error: recv got EOF from 127.31.59.131:33665 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:21 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:21.517428 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:21.508540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:33811)
0808 13:11:21.509132 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:21.509147 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:21.509272 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:21.509604 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:21.509607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:21.510020 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:21.510212 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.510222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.511392 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.511398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.512343 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:21.512356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.513101 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.513108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:21.513267 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.513888 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:21.513907 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:21.517093 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":314,"threads_started":1}
W20260808 13:11:21.517596 9411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:21.508625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:33811 (local address 127.31.59.190:38483)
0808 13:11:21.508777 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:21.508782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:21.509208 (+ 426us) server_negotiation.cc:407] Connection header received
0808 13:11:21.509385 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:21.509388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:21.509490 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:21.509566 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:21.510339 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.511254 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.512490 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:21.512914 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:21.513022 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:21.517317 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:33811: BlockingRecv error: recv got EOF from 127.31.59.130:33811 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:22.080792 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:22.072676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:43071)
0808 13:11:22.073176 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:22.073193 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:22.073312 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:22.073605 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:22.073608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:22.074023 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:22.074175 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.074183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.075147 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.075151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.076051 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.076062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.076690 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.076693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.076812 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.077331 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:22.077348 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:22.080556 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":278,"threads_started":1}
W20260808 13:11:22.081025 9416 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:22.072755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43071 (local address 127.31.59.190:38483)
0808 13:11:22.073229 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:22.073233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:22.073249 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:11:22.073401 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:22.073404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:22.073483 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:22.073570 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:22.074283 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.075023 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.076170 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.076583 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.076672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.080757 (+ 4085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:43071: BlockingRecv error: recv got EOF from 127.31.59.129:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":270,"threads_started":1}
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:22.213984 9417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:22.205334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:60661)
0808 13:11:22.205906 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:22.205923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:22.206052 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:22.206325 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:22.206328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:22.206723 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:22.206917 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.206926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.207865 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.207869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.208801 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.208811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.209404 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.209408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.209578 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.210112 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:22.210129 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:22.213751 (+ 3622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":312,"threads_started":1}
W20260808 13:11:22.214082 9416 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:22.205372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:60661 (local address 127.31.59.190:38483)
0808 13:11:22.205609 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:22.205616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:22.205999 (+ 383us) server_negotiation.cc:407] Connection header received
0808 13:11:22.206143 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:22.206147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:22.206211 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:22.206294 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:22.207023 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.207758 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.208898 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.209275 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.209358 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.213903 (+ 4545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:60661: BlockingRecv error: recv got EOF from 127.31.59.131:60661 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:22 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:22.527195 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:22.518420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:54627)
0808 13:11:22.519008 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:22.519023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:22.519150 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:22.519532 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:22.519537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:22.520000 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:22.520155 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.520164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.521195 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.521199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.522184 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:22.522196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.522842 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.522848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:22.523012 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.523541 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:22.523558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:22.526962 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":388,"threads_started":1}
W20260808 13:11:22.527381 9416 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:22.518652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54627 (local address 127.31.59.190:38483)
0808 13:11:22.518844 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:22.518850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:22.519104 (+ 254us) server_negotiation.cc:407] Connection header received
0808 13:11:22.519293 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:22.519297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:22.519385 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:22.519471 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:22.520301 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.521059 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.522321 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:22.522707 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:22.522813 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:22.527145 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:54627: BlockingRecv error: recv got EOF from 127.31.59.130:54627 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.129@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:23.090859 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:23.082005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.129:52453)
0808 13:11:23.082490 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:23.082507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:23.082619 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:23.082967 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:23.082973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:23.083391 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:23.083617 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.083627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.084617 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.084621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.085755 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.085769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.086415 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.086420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.086575 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.087198 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:23.087215 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:23.090583 (+ 3368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
W20260808 13:11:23.091034 9420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:23.082059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:52453 (local address 127.31.59.190:38483)
0808 13:11:23.082512 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:23.082517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:23.082577 (+ 60us) server_negotiation.cc:407] Connection header received
0808 13:11:23.082702 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:23.082706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:23.082780 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:23.082867 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:23.083760 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.084478 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.085872 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.086290 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.086377 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.090789 (+ 4412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.129:52453: BlockingRecv error: recv got EOF from 127.31.59.129:52453 (error 108)
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":239,"threads_started":1}
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.131@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:23.223997 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:23.214958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.131:58827)
0808 13:11:23.215514 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:23.215530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:23.215678 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:23.216003 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:23.216008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:23.216509 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:23.216667 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.216676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.217771 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.217775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.218699 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.218711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.219380 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.219386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.219548 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.220210 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:23.220227 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:23.223766 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":315,"threads_started":1}
W20260808 13:11:23.224151 9420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:23.215063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:58827 (local address 127.31.59.190:38483)
0808 13:11:23.215410 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:23.215417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:23.215631 (+ 214us) server_negotiation.cc:407] Connection header received
0808 13:11:23.215799 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:23.215805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:23.215870 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:23.215952 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:23.216817 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.217631 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.218842 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.219239 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.219351 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.223936 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.131:58827: BlockingRecv error: recv got EOF from 127.31.59.131:58827 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:11:23 dist-test-slave-49h3 krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.130@KRBTEST.COM for oryx/127.31.59.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:11:23.537418 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:23.528419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38483 (local address 127.31.59.130:59675)
0808 13:11:23.528933 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:23.528949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:23.529084 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:23.529419 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:23.529424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:23.529919 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:23.530149 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.530161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.531564 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.531568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.532508 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:23.532521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.533170 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.533174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:23.533303 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.533849 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:23.533866 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:23.537165 (+ 3299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.190:38483: Server oryx/127.31.59.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
W20260808 13:11:23.537611 9420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:23.528592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:59675 (local address 127.31.59.190:38483)
0808 13:11:23.528825 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:23.528831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:23.529031 (+ 200us) server_negotiation.cc:407] Connection header received
0808 13:11:23.529211 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:23.529215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:23.529290 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:23.529371 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:23.530295 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.531374 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.532658 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.533050 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:23.533151 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:23.537350 (+ 4199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.130:59675: BlockingRecv error: recv got EOF from 127.31.59.130:59675 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 13:11:23.994410 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8928
I20260808 13:11:24.006529 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:23.995495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.31.59.129:38829)
0808 13:11:23.996179 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:23.996187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:23.996212 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:11:23.996273 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:23.996280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:23.996511 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:23.996671 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:23.998198 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:23.999277 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.000448 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.000997 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.001089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:24.003489 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:24.003499 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:24.003513 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:24.003569 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:24.006002 (+ 2433us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:11:24.006297 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35704: Request ticket server kudu/127.31.59.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":378,"threads_started":1}
W20260808 13:11:24.006642 9424 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35704: Request ticket server kudu/127.31.59.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 13:11:24.006738 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8928: Not authorized: Client connection negotiation failed: client connection to 127.31.59.129:38829: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:11:24.006955 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8928
I20260808 13:11:24.026942 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9085
I20260808 13:11:24.039783 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:24.028017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35524 (local address 127.31.59.130:34933)
0808 13:11:24.028587 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:24.028594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:24.028745 (+ 151us) server_negotiation.cc:407] Connection header received
0808 13:11:24.028921 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:24.028928 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:24.029173 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:24.029373 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:24.030828 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.032005 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.033229 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.033906 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.034026 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:24.036480 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:24.036489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:24.036503 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:24.036541 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:24.039104 (+ 2563us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:11:24.039475 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35524: Request ticket server kudu/127.31.59.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":243,"threads_started":1}
W20260808 13:11:24.039824 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9085: Not authorized: Client connection negotiation failed: client connection to 127.31.59.130:34933: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 13:11:24.039904 9425 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35524: Request ticket server kudu/127.31.59.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:11:24.039991 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9085
I20260808 13:11:24.058578 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9226
I20260808 13:11:24.070932 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:24.059724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.31.59.131:33379)
0808 13:11:24.060437 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:24.060448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:24.060479 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:11:24.060568 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:24.060578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:24.060898 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:24.061117 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:24.062439 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.063471 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.064630 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.065188 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.065281 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:24.067741 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:24.067749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:24.067763 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:24.067810 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:24.070354 (+ 2544us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:11:24.070645 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58004: Request ticket server kudu/127.31.59.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":311,"threads_started":1}
W20260808 13:11:24.071033 9426 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58004: Request ticket server kudu/127.31.59.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 13:11:24.070996 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9226: Not authorized: Client connection negotiation failed: client connection to 127.31.59.131:33379: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:11:24.071153 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9226
I20260808 13:11:24.089073 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 8852
I20260808 13:11:24.090406 8920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:24.211956 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 8852
2026-08-08T13:11:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21476 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T13:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:24Z Disabled control of system clock
I20260808 13:11:24.298789 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:41037
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:41037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 13:11:24.629825 9432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:24.630132 9432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:24.630256 9432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:24.637790 9432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:24.637912 9432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:24.637979 9432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:24.638197 9432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:24.638286 9432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:24.647981 9432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:41037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:41037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:24.649358 9432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:24.651247 9432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:24.663091 9440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:24.663152 9437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:24.663129 9438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:24.663913 9432 server_base.cc:1034] running on GCE node
I20260808 13:11:24.664710 9432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:24.666213 9432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:24.667412 9432 hybrid_clock.cc:648] HybridClock initialized: now 1786194684667358 us; error 43 us; skew 500 ppm
I20260808 13:11:24.667761 9432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:24.670125 9432 webserver.cc:460] Webserver started at http://127.31.59.190:37347/ using document root <none> and password file <none>
I20260808 13:11:24.670830 9432 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:24.670930 9432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:24.671262 9432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:24.674289 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "ea6c21c71a714c03974cd1a85668afb0"
format_stamp: "Formatted at 2026-08-08 13:11:24 on dist-test-slave-49h3"
server_key: "0fdc5d89b398f4a4a7c20b1a2c1731f8"
server_key_iv: "97c0fde3995132996b2894b03902a49a"
server_key_version: "encryptionkey@0"
I20260808 13:11:24.675055 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "ea6c21c71a714c03974cd1a85668afb0"
format_stamp: "Formatted at 2026-08-08 13:11:24 on dist-test-slave-49h3"
server_key: "0fdc5d89b398f4a4a7c20b1a2c1731f8"
server_key_iv: "97c0fde3995132996b2894b03902a49a"
server_key_version: "encryptionkey@0"
I20260808 13:11:24.680080 9432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:11:24.683447 9447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:24.684727 9432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:11:24.684890 9432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "ea6c21c71a714c03974cd1a85668afb0"
format_stamp: "Formatted at 2026-08-08 13:11:24 on dist-test-slave-49h3"
server_key: "0fdc5d89b398f4a4a7c20b1a2c1731f8"
server_key_iv: "97c0fde3995132996b2894b03902a49a"
server_key_version: "encryptionkey@0"
I20260808 13:11:24.685084 9432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:24.707461 9432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:24.708356 9432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:24.708638 9432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:24.726660 9432 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:41037
I20260808 13:11:24.726665 9500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:41037 every 8 connection(s)
I20260808 13:11:24.728009 9432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:11:24.731712 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:24.733115 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9432
I20260808 13:11:24.733317 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:11:24.733992 31982 external_mini_cluster.cc:1428] Setting key 25f677a399b2de8e8de82130063d1bd2
I20260808 13:11:24.744014 9501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0: Bootstrap starting.
I20260808 13:11:24.747857 9501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:24.748147 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:24.736038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.31.59.190:41037)
0808 13:11:24.737062 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:24.737072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:24.737118 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:11:24.738859 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:24.738882 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:24.739333 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:24.739648 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:24.740368 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.741770 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.743653 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:24.744381 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:24.744514 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:24.745789 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:24.745801 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:24.745814 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:24.746365 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:24.746467 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:24.746487 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:24.746577 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:24.746856 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 13:11:24.747147 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":383,"threads_started":1}
I20260808 13:11:24.749221 9501 log.cc:824] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:24.752321 9501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0: No bootstrap required, opened a new log
I20260808 13:11:24.759603 9501 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } }
I20260808 13:11:24.759981 9501 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:24.760095 9501 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6c21c71a714c03974cd1a85668afb0, State: Initialized, Role: FOLLOWER
I20260808 13:11:24.760628 9501 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } }
I20260808 13:11:24.760843 9501 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:11:24.760967 9501 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:11:24.761119 9501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:24.763396 9501 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } }
I20260808 13:11:24.763935 9501 leader_election.cc:302] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea6c21c71a714c03974cd1a85668afb0; no voters:
I20260808 13:11:24.764601 9501 leader_election.cc:288] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:11:24.764709 9506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:24.765435 9506 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [term 1 LEADER]: Becoming Leader. State: Replica: ea6c21c71a714c03974cd1a85668afb0, State: Running, Role: LEADER
I20260808 13:11:24.766095 9506 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } }
I20260808 13:11:24.766863 9501 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:11:24.768925 9508 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea6c21c71a714c03974cd1a85668afb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } } }
I20260808 13:11:24.769378 9508 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:24.769886 9507 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea6c21c71a714c03974cd1a85668afb0. Latest consensus state: current_term: 1 leader_uuid: "ea6c21c71a714c03974cd1a85668afb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6c21c71a714c03974cd1a85668afb0" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 41037 } } }
I20260808 13:11:24.770305 9507 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:24.772722 9514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:11:24.777100 9514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:11:24.786516 9514 catalog_manager.cc:1347] Generated new cluster ID: 12e3ff6986d1486583cbab3fbd4dc17a
I20260808 13:11:24.786648 9514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:11:24.797487 9514 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:11:24.798771 9514 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:11:24.808614 9514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea6c21c71a714c03974cd1a85668afb0: Generated new TSK 0
I20260808 13:11:24.809387 9514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:11:24.822669 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--builtin_ntp_servers=127.31.59.148:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 13:11:25.158207 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:25.158507 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:25.158614 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:25.165872 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:25.165980 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:25.166121 9525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:11:25.175221 9525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:25.176676 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:25.178673 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:25.191699 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:25.191799 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:25.191732 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:25.192519 9525 server_base.cc:1034] running on GCE node
I20260808 13:11:25.193099 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:25.194072 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:25.195274 9525 hybrid_clock.cc:648] HybridClock initialized: now 1786194685195217 us; error 47 us; skew 500 ppm
I20260808 13:11:25.195574 9525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:25.198905 9525 webserver.cc:460] Webserver started at http://127.31.59.129:42247/ using document root <none> and password file <none>
I20260808 13:11:25.199774 9525 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:25.199889 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:25.200222 9525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:25.203152 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "ec8a6c41abb74b7cb70a8576455b8d87"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "b4d963c2cc0cfd490646b2210ddabd4e"
server_key_iv: "2a44b3081ec9411af6a44300bc88ae38"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.203943 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "ec8a6c41abb74b7cb70a8576455b8d87"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "b4d963c2cc0cfd490646b2210ddabd4e"
server_key_iv: "2a44b3081ec9411af6a44300bc88ae38"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.208968 9525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:11:25.212314 9539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.213590 9525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:11:25.213744 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "ec8a6c41abb74b7cb70a8576455b8d87"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "b4d963c2cc0cfd490646b2210ddabd4e"
server_key_iv: "2a44b3081ec9411af6a44300bc88ae38"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.213940 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:25.242888 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:25.243825 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:25.244140 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:25.245113 9525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:25.246660 9525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:25.246755 9525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.246865 9525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:25.246937 9525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.278534 9525 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:42953
I20260808 13:11:25.278604 9659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:42953 every 8 connection(s)
I20260808 13:11:25.279824 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:11:25.290305 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9525
I20260808 13:11:25.290522 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:11:25.291230 31982 external_mini_cluster.cc:1428] Setting key 9ef349e8e626d7632c6c980b27f09764
I20260808 13:11:25.292060 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:25.282290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:50425 (local address 127.31.59.190:41037)
0808 13:11:25.282744 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:25.282749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:25.283930 (+ 1181us) server_negotiation.cc:407] Connection header received
0808 13:11:25.285687 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:25.285692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:25.285779 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:25.285921 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:25.286891 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.287600 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.289304 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.289764 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.289901 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:25.291262 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:25.291269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:25.291273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:25.291297 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:25.291353 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:25.291371 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:25.291461 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:25.291660 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 13:11:25.291763 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":208,"threads_started":1}
I20260808 13:11:25.293324 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:25.282880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:41037 (local address 127.31.59.129:50425)
0808 13:11:25.283770 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:25.283826 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:25.285420 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:25.286085 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:25.286092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:25.286174 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:25.286746 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:25.286764 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.287750 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.287753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:25.289171 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:25.289185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.289973 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.289978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:25.290192 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:25.290907 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:25.290910 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:25.290951 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:25.291052 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:25.291459 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:25.291468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:25.291475 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:11:25.291621 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 13:11:25.291985 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":270,"threads_started":1}
I20260808 13:11:25.294358 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--builtin_ntp_servers=127.31.59.148:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 13:11:25.295970 9661 heartbeater.cc:347] Connected to a master server at 127.31.59.190:41037
I20260808 13:11:25.296278 9661 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:25.297083 9661 heartbeater.cc:511] Master 127.31.59.190:41037 requested a full tablet report, sending...
I20260808 13:11:25.299350 9464 ts_manager.cc:194] Registered new tserver with Master: ec8a6c41abb74b7cb70a8576455b8d87 (127.31.59.129:42953)
I20260808 13:11:25.301191 9464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.129:50425
W20260808 13:11:25.630431 9665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:25.630704 9665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:25.630800 9665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:25.637957 9665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:25.638056 9665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:25.638167 9665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:11:25.647012 9665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:25.648239 9665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:25.650242 9665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:25.663115 9670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:25.663182 9671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:25.663358 9673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:25.664016 9665 server_base.cc:1034] running on GCE node
I20260808 13:11:25.664623 9665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:25.665603 9665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:25.666814 9665 hybrid_clock.cc:648] HybridClock initialized: now 1786194685666748 us; error 53 us; skew 500 ppm
I20260808 13:11:25.667137 9665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:25.669490 9665 webserver.cc:460] Webserver started at http://127.31.59.130:36197/ using document root <none> and password file <none>
I20260808 13:11:25.670233 9665 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:25.670347 9665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:25.670677 9665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:25.673659 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "69eb3bb4df7f4a47a9108b4eff041db1"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "2a085f09f5e4768693af189706e10d58"
server_key_iv: "1ca02d01cbe30bafc336480c51470d95"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.674475 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "69eb3bb4df7f4a47a9108b4eff041db1"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "2a085f09f5e4768693af189706e10d58"
server_key_iv: "1ca02d01cbe30bafc336480c51470d95"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.679768 9665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260808 13:11:25.683156 9679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.684368 9665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:11:25.684517 9665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "69eb3bb4df7f4a47a9108b4eff041db1"
format_stamp: "Formatted at 2026-08-08 13:11:25 on dist-test-slave-49h3"
server_key: "2a085f09f5e4768693af189706e10d58"
server_key_iv: "1ca02d01cbe30bafc336480c51470d95"
server_key_version: "encryptionkey@0"
I20260808 13:11:25.684711 9665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:25.712208 9665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:25.713233 9665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:25.713588 9665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:25.714593 9665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:25.716078 9665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:25.716179 9665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.716295 9665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:25.716378 9665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:25.749245 9665 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44559
I20260808 13:11:25.749315 9799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44559 every 8 connection(s)
I20260808 13:11:25.750581 9665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:11:25.751011 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9665
I20260808 13:11:25.751185 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:11:25.751734 31982 external_mini_cluster.cc:1428] Setting key 00227523dfce5cacb98532bd2ccb2772
I20260808 13:11:25.754854 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--builtin_ntp_servers=127.31.59.148:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 13:11:25.764237 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:25.753233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:52547 (local address 127.31.59.190:41037)
0808 13:11:25.753424 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:25.753430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:25.754870 (+ 1440us) server_negotiation.cc:407] Connection header received
0808 13:11:25.756906 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:25.756911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:25.756990 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:25.757080 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:25.758383 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.759328 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.761322 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.761858 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.761976 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:25.763395 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:25.763400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:25.763404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:25.763427 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:25.763466 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:25.763481 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:25.763561 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:25.763857 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 13:11:25.763957 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:11:25.765475 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:25.753790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:41037 (local address 127.31.59.130:52547)
0808 13:11:25.754718 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:25.754753 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:25.756638 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:25.757414 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:25.757423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:25.757552 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:25.758199 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:25.758219 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.759474 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.759484 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:25.761162 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:25.761177 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:25.762057 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:25.762063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:25.762327 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:25.763061 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:25.763065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:25.763113 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:25.763207 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:25.763616 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:25.763629 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:25.763637 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:11:25.763818 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 13:11:25.764197 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":306,"threads_started":1}
I20260808 13:11:25.766995 9800 heartbeater.cc:347] Connected to a master server at 127.31.59.190:41037
I20260808 13:11:25.767287 9800 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:25.768175 9800 heartbeater.cc:511] Master 127.31.59.190:41037 requested a full tablet report, sending...
I20260808 13:11:25.769667 9464 ts_manager.cc:194] Registered new tserver with Master: 69eb3bb4df7f4a47a9108b4eff041db1 (127.31.59.130:44559)
I20260808 13:11:25.770762 9464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.130:52547
W20260808 13:11:26.091032 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:26.091317 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:26.091408 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:26.098646 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:26.098757 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:26.098874 9803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:11:26.107960 9803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:41037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:26.109196 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:26.110968 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:26.123860 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:26.123941 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:26.123898 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:26.124706 9803 server_base.cc:1034] running on GCE node
I20260808 13:11:26.125306 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:26.126279 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:26.127475 9803 hybrid_clock.cc:648] HybridClock initialized: now 1786194686127430 us; error 34 us; skew 500 ppm
I20260808 13:11:26.127789 9803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:26.129971 9803 webserver.cc:460] Webserver started at http://127.31.59.131:38969/ using document root <none> and password file <none>
I20260808 13:11:26.130656 9803 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:26.130754 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:26.131083 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:26.133962 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "9a26f26c7d3b4125ac7d29b905c623fa"
format_stamp: "Formatted at 2026-08-08 13:11:26 on dist-test-slave-49h3"
server_key: "28ff3d5e6550c746cbaabd6e5aac1dc5"
server_key_iv: "704f006f61841b8711f12e820080b623"
server_key_version: "encryptionkey@0"
I20260808 13:11:26.134718 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "9a26f26c7d3b4125ac7d29b905c623fa"
format_stamp: "Formatted at 2026-08-08 13:11:26 on dist-test-slave-49h3"
server_key: "28ff3d5e6550c746cbaabd6e5aac1dc5"
server_key_iv: "704f006f61841b8711f12e820080b623"
server_key_version: "encryptionkey@0"
I20260808 13:11:26.139976 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:11:26.143415 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:26.144677 9803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 13:11:26.144852 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "9a26f26c7d3b4125ac7d29b905c623fa"
format_stamp: "Formatted at 2026-08-08 13:11:26 on dist-test-slave-49h3"
server_key: "28ff3d5e6550c746cbaabd6e5aac1dc5"
server_key_iv: "704f006f61841b8711f12e820080b623"
server_key_version: "encryptionkey@0"
I20260808 13:11:26.145059 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:26.170358 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:26.171277 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:26.171584 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:26.172588 9803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:26.174175 9803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:26.174278 9803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:26.174388 9803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:26.174460 9803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:26.207196 9803 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:40913
I20260808 13:11:26.207273 9935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:40913 every 8 connection(s)
I20260808 13:11:26.208503 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:11:26.214027 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9803
I20260808 13:11:26.214210 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:11:26.214624 31982 external_mini_cluster.cc:1428] Setting key 02d517744f7aed6ce1809744708637ef
I20260808 13:11:26.225330 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:26.211016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:33731 (local address 127.31.59.190:41037)
0808 13:11:26.211196 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:26.211201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:26.212580 (+ 1379us) server_negotiation.cc:407] Connection header received
0808 13:11:26.214540 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:26.214546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:26.214622 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:26.214708 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:26.215743 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.216505 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.218192 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.218687 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.218807 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:26.224496 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:26.224501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:26.224504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:26.224523 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:26.224556 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:26.224568 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:26.224644 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:26.224884 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 13:11:26.224991 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:11:26.226449 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:26.211547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:41037 (local address 127.31.59.131:33731)
0808 13:11:26.212430 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:26.212471 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:26.214271 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:26.214935 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:26.214943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:26.215031 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:26.215583 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:26.215599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.216660 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.216664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:26.218054 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:26.218066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.218910 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.218917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:26.219141 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:26.224129 (+ 4988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:26.224132 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:26.224179 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:26.224286 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:26.224659 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:26.224671 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:26.224679 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:11:26.224842 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 13:11:26.225285 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260808 13:11:26.227792 9936 heartbeater.cc:347] Connected to a master server at 127.31.59.190:41037
I20260808 13:11:26.228065 9936 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:26.228925 9936 heartbeater.cc:511] Master 127.31.59.190:41037 requested a full tablet report, sending...
I20260808 13:11:26.230362 9464 ts_manager.cc:194] Registered new tserver with Master: 9a26f26c7d3b4125ac7d29b905c623fa (127.31.59.131:40913)
I20260808 13:11:26.231150 9464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.131:33731
I20260808 13:11:26.242002 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:11:26.247888 9663 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:26.246332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.31.59.190:41037)
0808 13:11:26.246489 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:26.246493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:26.246996 (+ 503us) server_negotiation.cc:407] Connection header received
0808 13:11:26.247125 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:26.247128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:26.247203 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:26.247281 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:26.247600 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36422: BlockingRecv error: recv got EOF from 127.0.0.1:36422 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:11:26.248278 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9525
I20260808 13:11:26.255592 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:26.249214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.31.59.129:42953)
0808 13:11:26.249817 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:26.249824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:26.249851 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:11:26.249911 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:26.249917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:26.250122 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:26.250251 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:26.250556 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.251505 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.252665 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.253212 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.253315 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:26.254272 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:26.254280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:26.254292 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:26.254851 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:26.254897 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:26.254928 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:26.255010 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:26.255106 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 13:11:26.255221 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":331,"threads_started":1}
I20260808 13:11:26.256803 9655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:26.339397 9661 heartbeater.cc:503] Master 127.31.59.190:41037 was elected leader, sending a full tablet report...
I20260808 13:11:26.340509 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9525
W20260808 13:11:26.358736 9448 connection.cc:535] server connection from 127.31.59.129:50425 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:11:26.358978 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9665
I20260808 13:11:26.366554 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:26.359898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.31.59.130:44559)
0808 13:11:26.360414 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:26.360421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:26.360447 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:11:26.360508 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:26.360514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:26.360733 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:26.360854 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:26.361241 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.362308 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.363469 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.364031 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.364126 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:26.365089 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:26.365097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:26.365111 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:26.365765 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:26.365815 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:26.365836 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:26.365936 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:26.366056 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 13:11:26.366172 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":235,"threads_started":1}
I20260808 13:11:26.368002 9794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:26.449380 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9665
I20260808 13:11:26.469084 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9803
I20260808 13:11:26.476948 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:26.470125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.31.59.131:40913)
0808 13:11:26.470789 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:26.470796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:26.470827 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:11:26.470888 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:26.470894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:26.471109 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:26.471233 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:26.471644 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.472751 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.474007 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:26.474556 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:26.474655 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:26.475575 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:26.475582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:26.475594 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:26.476184 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:26.476231 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:26.476251 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:26.476347 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:26.476434 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 13:11:26.476545 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":344,"threads_started":1}
I20260808 13:11:26.478245 9931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:26.560573 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9803
I20260808 13:11:26.578392 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9432
I20260808 13:11:26.579643 9496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:26.685057 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9432
2026-08-08T13:11:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2469 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T13:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:26Z Disabled control of system clock
I20260808 13:11:26.770941 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45081
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 13:11:27.107303 9955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:27.107604 9955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:27.107697 9955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:27.115063 9955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:27.115180 9955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:27.115226 9955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:27.115283 9955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:27.115325 9955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:27.124603 9955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:27.125962 9955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:27.127897 9955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:27.139490 9960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:27.139549 9962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:27.139453 9964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:27.140818 9955 server_base.cc:1034] running on GCE node
I20260808 13:11:27.141649 9955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:27.143064 9955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:27.144309 9955 hybrid_clock.cc:648] HybridClock initialized: now 1786194687144251 us; error 45 us; skew 500 ppm
I20260808 13:11:27.144677 9955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:27.146986 9955 webserver.cc:460] Webserver started at http://127.31.59.190:36785/ using document root <none> and password file <none>
I20260808 13:11:27.147691 9955 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:27.147804 9955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:27.148123 9955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:27.151165 9955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "a31b9a8a211f44cd8d59c0e2c088c294"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "00b65ea2530377019b2e2b761799fe64"
server_key_iv: "6a52d234fe6ae4ba46b2d5f2631a25ab"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.151942 9955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "a31b9a8a211f44cd8d59c0e2c088c294"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "00b65ea2530377019b2e2b761799fe64"
server_key_iv: "6a52d234fe6ae4ba46b2d5f2631a25ab"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.157001 9955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260808 13:11:27.160329 9970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:27.161613 9955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:11:27.161787 9955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "a31b9a8a211f44cd8d59c0e2c088c294"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "00b65ea2530377019b2e2b761799fe64"
server_key_iv: "6a52d234fe6ae4ba46b2d5f2631a25ab"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.161988 9955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:27.167629 9955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:27.168418 9955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:27.168696 9955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:27.186200 9955 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:45081
I20260808 13:11:27.186220 10024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:45081 every 8 connection(s)
I20260808 13:11:27.187512 9955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:11:27.188046 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 9955
I20260808 13:11:27.188221 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:11:27.188992 31982 external_mini_cluster.cc:1428] Setting key 2a9c748879295d2bb104015c3db3d44e
I20260808 13:11:27.191738 10025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:27.197072 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:27.190996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.31.59.190:45081)
0808 13:11:27.191991 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:27.192003 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:27.192044 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:11:27.193662 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:27.193681 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:27.194081 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:27.194415 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:27.195470 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:27.195476 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:27.195488 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:27.195502 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:27.195605 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:27.195620 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:27.195843 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:27.195999 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 13:11:27.196224 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":365,"threads_started":1}
I20260808 13:11:27.202780 10025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294: Bootstrap starting.
I20260808 13:11:27.206423 10025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:27.207499 10025 log.cc:824] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:27.210307 10025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294: No bootstrap required, opened a new log
I20260808 13:11:27.217353 10025 raft_consensus.cc:348] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } }
I20260808 13:11:27.217706 10025 raft_consensus.cc:374] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:27.217782 10025 raft_consensus.cc:729] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31b9a8a211f44cd8d59c0e2c088c294, State: Initialized, Role: FOLLOWER
I20260808 13:11:27.218250 10025 consensus_queue.cc:260] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } }
I20260808 13:11:27.218449 10025 raft_consensus.cc:388] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:11:27.218540 10025 raft_consensus.cc:482] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:11:27.218672 10025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:27.221311 10025 raft_consensus.cc:504] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } }
I20260808 13:11:27.221848 10025 leader_election.cc:302] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a31b9a8a211f44cd8d59c0e2c088c294; no voters:
I20260808 13:11:27.222405 10025 leader_election.cc:288] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:11:27.222579 10030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:27.223333 10030 raft_consensus.cc:686] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [term 1 LEADER]: Becoming Leader. State: Replica: a31b9a8a211f44cd8d59c0e2c088c294, State: Running, Role: LEADER
I20260808 13:11:27.223965 10030 consensus_queue.cc:237] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } }
I20260808 13:11:27.224566 10025 sys_catalog.cc:564] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:11:27.226697 10032 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } } }
I20260808 13:11:27.226703 10031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a31b9a8a211f44cd8d59c0e2c088c294. Latest consensus state: current_term: 1 leader_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31b9a8a211f44cd8d59c0e2c088c294" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45081 } } }
I20260808 13:11:27.227283 10031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:27.227659 10032 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294 [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:27.228946 10039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:11:27.234263 10039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:11:27.242940 10039 catalog_manager.cc:1347] Generated new cluster ID: d2895344b0ce40fdb9cba5ae4e292835
I20260808 13:11:27.243068 10039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:11:27.257256 10039 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:11:27.257809 10039 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:11:27.274243 10039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a31b9a8a211f44cd8d59c0e2c088c294: Generated new TSK 0
I20260808 13:11:27.275327 10039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:11:27.291723 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--builtin_ntp_servers=127.31.59.148:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 13:11:27.629496 10049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:27.629808 10049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:27.629917 10049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:27.637108 10049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:27.637212 10049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:27.637351 10049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:11:27.646437 10049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:27.647729 10049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:27.649621 10049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:27.662521 10055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:27.662551 10058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:27.662598 10056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:27.663372 10049 server_base.cc:1034] running on GCE node
I20260808 13:11:27.663959 10049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:27.664883 10049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:27.666149 10049 hybrid_clock.cc:648] HybridClock initialized: now 1786194687666096 us; error 36 us; skew 500 ppm
I20260808 13:11:27.666448 10049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:27.668777 10049 webserver.cc:460] Webserver started at http://127.31.59.129:33921/ using document root <none> and password file <none>
I20260808 13:11:27.669476 10049 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:27.669579 10049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:27.669867 10049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:27.672914 10049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "e4133529d890457eb3e78716fe18f91f"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "6651106101d503d9b81c93c959db5582"
server_key_iv: "4f617f47ac98670fdf0d149801a2c3d6"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.673795 10049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "e4133529d890457eb3e78716fe18f91f"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "6651106101d503d9b81c93c959db5582"
server_key_iv: "4f617f47ac98670fdf0d149801a2c3d6"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.678699 10049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:11:27.681905 10064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:27.683127 10049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:11:27.683283 10049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "e4133529d890457eb3e78716fe18f91f"
format_stamp: "Formatted at 2026-08-08 13:11:27 on dist-test-slave-49h3"
server_key: "6651106101d503d9b81c93c959db5582"
server_key_iv: "4f617f47ac98670fdf0d149801a2c3d6"
server_key_version: "encryptionkey@0"
I20260808 13:11:27.683475 10049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:27.688925 10049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:27.689798 10049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:27.690044 10049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:27.690923 10049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:27.692319 10049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:27.692417 10049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:27.692521 10049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:27.692590 10049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:27.729514 10049 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:41961
I20260808 13:11:27.729570 10183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:41961 every 8 connection(s)
I20260808 13:11:27.731062 10049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:11:27.739167 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:27.733555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:39121 (local address 127.31.59.190:45081)
0808 13:11:27.734008 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:27.734012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:27.735298 (+ 1286us) server_negotiation.cc:407] Connection header received
0808 13:11:27.736995 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:27.737001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:27.737066 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:27.737162 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:27.738446 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:27.738452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:27.738457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:27.738474 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:27.738517 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:27.738528 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:27.738639 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:27.738820 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 13:11:27.738885 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":227,"threads_started":1}
I20260808 13:11:27.739449 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10049
I20260808 13:11:27.739617 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:11:27.740135 10187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:27.734153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45081 (local address 127.31.59.129:39121)
0808 13:11:27.735149 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:27.735187 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:27.736736 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:27.737415 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:27.737423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:27.737563 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:27.738155 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:27.738157 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:27.738193 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:27.738285 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:27.738617 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:27.738628 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:27.738635 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:11:27.738888 (+ 253us) client_negotiation.cc:240] Negotiation successful
0808 13:11:27.739156 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":278,"threads_started":1}
I20260808 13:11:27.740265 31982 external_mini_cluster.cc:1428] Setting key 4c7b3a4b2bff29f39236b9e373f17fa8
I20260808 13:11:27.741720 10184 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45081
I20260808 13:11:27.742034 10184 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:27.742604 10184 heartbeater.cc:511] Master 127.31.59.190:45081 requested a full tablet report, sending...
I20260808 13:11:27.743069 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--builtin_ntp_servers=127.31.59.148:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 13:11:27.744501 9987 ts_manager.cc:194] Registered new tserver with Master: e4133529d890457eb3e78716fe18f91f (127.31.59.129:41961)
W20260808 13:11:28.080462 10188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:28.080741 10188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:28.080873 10188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:28.090556 10188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:28.090703 10188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:28.090896 10188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:11:28.102708 10188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:28.104005 10188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:28.105818 10188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:28.118853 10196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:28.118891 10194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:28.119098 10193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:28.119630 10188 server_base.cc:1034] running on GCE node
I20260808 13:11:28.120244 10188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:28.121133 10188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:28.122329 10188 hybrid_clock.cc:648] HybridClock initialized: now 1786194688122269 us; error 46 us; skew 500 ppm
I20260808 13:11:28.122648 10188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:28.124912 10188 webserver.cc:460] Webserver started at http://127.31.59.130:34299/ using document root <none> and password file <none>
I20260808 13:11:28.125684 10188 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:28.125790 10188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:28.126130 10188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:28.129092 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "bd02b88eb60f4a6d8fdd9b725468b1cf"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "fd55c9f79824212fdb4242a3cfcad9dc"
server_key_iv: "c41013c1b42569029126fdfced213752"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.129935 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "bd02b88eb60f4a6d8fdd9b725468b1cf"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "fd55c9f79824212fdb4242a3cfcad9dc"
server_key_iv: "c41013c1b42569029126fdfced213752"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.135123 10188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:11:28.138623 10205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.139963 10188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:11:28.140147 10188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "bd02b88eb60f4a6d8fdd9b725468b1cf"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "fd55c9f79824212fdb4242a3cfcad9dc"
server_key_iv: "c41013c1b42569029126fdfced213752"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.140360 10188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:28.146052 10188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:28.146849 10188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:28.147102 10188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:28.148011 10188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:28.149421 10188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:28.149574 10188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.149683 10188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:28.149755 10188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.181624 10188 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:39897
I20260808 13:11:28.181659 10323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:39897 every 8 connection(s)
I20260808 13:11:28.182942 10188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:11:28.190748 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10188
I20260808 13:11:28.190918 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:11:28.191485 31982 external_mini_cluster.cc:1428] Setting key d77fe3ddb20e0b05f1686889e5e0f3f6
I20260808 13:11:28.191658 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.185665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:39197 (local address 127.31.59.190:45081)
0808 13:11:28.185907 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.185914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.187450 (+ 1536us) server_negotiation.cc:407] Connection header received
0808 13:11:28.189173 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.189180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.189246 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.189354 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.190872 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:28.190879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:28.190884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:28.190905 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:28.190949 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:28.190963 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:28.191081 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:28.191305 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:11:28.191371 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:11:28.192830 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.186224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45081 (local address 127.31.59.130:39197)
0808 13:11:28.187279 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:28.187325 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:28.188907 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:28.189603 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:28.189611 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:28.189723 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:28.190477 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:28.190480 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:28.190543 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:28.190680 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:28.191073 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:28.191084 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:28.191092 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:11:28.191389 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 13:11:28.191681 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":310,"threads_started":1}
I20260808 13:11:28.194382 10324 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45081
I20260808 13:11:28.194543 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--builtin_ntp_servers=127.31.59.148:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 13:11:28.194712 10324 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:28.195446 10324 heartbeater.cc:511] Master 127.31.59.190:45081 requested a full tablet report, sending...
I20260808 13:11:28.196969 9987 ts_manager.cc:194] Registered new tserver with Master: bd02b88eb60f4a6d8fdd9b725468b1cf (127.31.59.130:39897)
W20260808 13:11:28.524669 10327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:28.524941 10327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:28.525086 10327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:28.532541 10327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:28.532655 10327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:28.532811 10327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:11:28.542112 10327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:28.543457 10327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:28.545279 10327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:28.558126 10333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:28.558091 10332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:28.558284 10335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:28.558926 10327 server_base.cc:1034] running on GCE node
I20260808 13:11:28.559479 10327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:28.560381 10327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:28.561530 10327 hybrid_clock.cc:648] HybridClock initialized: now 1786194688561485 us; error 37 us; skew 500 ppm
I20260808 13:11:28.561829 10327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:11:28.563992 10327 webserver.cc:460] Webserver started at http://127.31.59.131:33279/ using document root <none> and password file <none>
I20260808 13:11:28.564667 10327 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:28.564766 10327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:28.565073 10327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:28.568135 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "f3d2d38cf62b4c05be49c9b328bf2f9a"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "971c33d479efef4190908968bec20a91"
server_key_iv: "58c6be00431750ee7b0a2248734454fb"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.568956 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "f3d2d38cf62b4c05be49c9b328bf2f9a"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "971c33d479efef4190908968bec20a91"
server_key_iv: "58c6be00431750ee7b0a2248734454fb"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.574106 10327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:11:28.577312 10341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.578598 10327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:11:28.578747 10327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "f3d2d38cf62b4c05be49c9b328bf2f9a"
format_stamp: "Formatted at 2026-08-08 13:11:28 on dist-test-slave-49h3"
server_key: "971c33d479efef4190908968bec20a91"
server_key_iv: "58c6be00431750ee7b0a2248734454fb"
server_key_version: "encryptionkey@0"
I20260808 13:11:28.578943 10327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:28.584257 10327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:28.584986 10327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:28.585233 10327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:28.586148 10327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:28.587531 10327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:28.587626 10327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.587731 10327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:28.587803 10327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:28.616273 10327 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:33759
I20260808 13:11:28.616314 10458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:33759 every 8 connection(s)
I20260808 13:11:28.617555 10327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:11:28.624154 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10327
I20260808 13:11:28.624365 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:11:28.625015 31982 external_mini_cluster.cc:1428] Setting key bd3619fe53c5c56bbabaa34294e820bb
I20260808 13:11:28.625397 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.619916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:49179 (local address 127.31.59.190:45081)
0808 13:11:28.620096 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.620100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.621415 (+ 1315us) server_negotiation.cc:407] Connection header received
0808 13:11:28.623051 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.623055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.623112 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.623197 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.624500 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:28.624507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:28.624511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:28.624533 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:28.624577 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:28.624594 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:28.624688 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:28.625028 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 13:11:28.625091 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:11:28.626478 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.620408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45081 (local address 127.31.59.131:49179)
0808 13:11:28.621267 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:28.621301 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:28.622825 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:28.623366 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:28.623373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:28.623459 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:28.624106 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:11:28.624109 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:28.624164 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:11:28.624279 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:28.624796 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:28.624806 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:28.624814 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:11:28.625091 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 13:11:28.625352 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":263,"threads_started":1}
I20260808 13:11:28.627737 10459 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45081
I20260808 13:11:28.628006 10459 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:28.628630 10459 heartbeater.cc:511] Master 127.31.59.190:45081 requested a full tablet report, sending...
I20260808 13:11:28.629987 9987 ts_manager.cc:194] Registered new tserver with Master: f3d2d38cf62b4c05be49c9b328bf2f9a (127.31.59.131:33759)
I20260808 13:11:28.640236 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:11:28.646232 10186 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:11:28.644677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.31.59.190:45081)
0808 13:11:28.644846 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.644850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.645352 (+ 502us) server_negotiation.cc:407] Connection header received
0808 13:11:28.645521 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.645524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.645578 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.645658 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.646020 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43878: BlockingRecv error: recv got EOF from 127.0.0.1:43878 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:11:28.647073 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10049
I20260808 13:11:28.650591 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.648111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.31.59.129:41961)
0808 13:11:28.648641 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.648647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.648673 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:11:28.648731 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.648738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.648943 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.649062 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.649957 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:28.649963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:28.649974 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:28.649993 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:28.650031 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:28.650044 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:28.650144 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:28.650205 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 13:11:28.650301 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":256,"threads_started":1}
I20260808 13:11:28.651882 10179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:28.734254 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10049
I20260808 13:11:28.753602 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10188
I20260808 13:11:28.757239 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.754562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.31.59.130:39897)
0808 13:11:28.755065 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.755072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.755098 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:11:28.755162 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.755168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.755399 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.755544 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.756527 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:28.756534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:28.756545 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:28.756564 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:28.756602 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:28.756617 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:28.756739 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:28.756883 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 13:11:28.756987 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":233,"threads_started":1}
I20260808 13:11:28.758605 10319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:28.840325 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10188
I20260808 13:11:28.858214 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10327
I20260808 13:11:28.862015 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:28.859283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45860 (local address 127.31.59.131:33759)
0808 13:11:28.859941 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:28.859947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:28.859979 (+ 32us) server_negotiation.cc:407] Connection header received
0808 13:11:28.860044 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:28.860051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:28.860284 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:28.860408 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:28.861286 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:28.861292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:28.861303 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:11:28.861319 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:28.861358 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:11:28.861373 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:28.861533 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:28.861591 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 13:11:28.861683 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":339,"threads_started":1}
I20260808 13:11:28.863304 10454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:28.945060 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10327
I20260808 13:11:28.961074 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 9955
I20260808 13:11:28.962294 10020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:29.067343 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 9955
2026-08-08T13:11:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2381 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:11:29 dist-test-slave-49h3 krb5kdc[10481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:11:29 dist-test-slave-49h3 krb5kdc[10481](info): set up 2 sockets
Aug 08 13:11:29 dist-test-slave-49h3 krb5kdc[10481](info): commencing operation
krb5kdc: starting...
W20260808 13:11:30.149863 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:11:30 dist-test-slave-49h3 krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:11:30.429893 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33057
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:30.759408 10495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:30.759719 10495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:30.759859 10495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:30.767366 10495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:11:30.767498 10495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:30.767572 10495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:30.767653 10495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:11:30.767712 10495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:11:30.778430 10495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:30.779894 10495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:30.781255 10495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:30.792994 10502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:30.793066 10501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:30.793021 10504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:30.793933 10495 server_base.cc:1034] running on GCE node
I20260808 13:11:30.794714 10495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:30.796141 10495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:30.797355 10495 hybrid_clock.cc:648] HybridClock initialized: now 1786194690797313 us; error 37 us; skew 500 ppm
I20260808 13:11:30.797719 10495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:30 dist-test-slave-49h3 krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:30.800737 10495 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:11:30.802309 10495 webserver.cc:460] Webserver started at http://127.31.59.190:34827/ using document root <none> and password file <none>
I20260808 13:11:30.802991 10495 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:30.803114 10495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:30.803434 10495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:30.806674 10495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "a26075cf6a1b4067b4a5abdf2e4d869e"
format_stamp: "Formatted at 2026-08-08 13:11:30 on dist-test-slave-49h3"
server_key: "f526740ffe7b696c387d4e09b3423e9d"
server_key_iv: "b8ea23601aa75cefa058aaf6836c54dc"
server_key_version: "encryptionkey@0"
I20260808 13:11:30.807451 10495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "a26075cf6a1b4067b4a5abdf2e4d869e"
format_stamp: "Formatted at 2026-08-08 13:11:30 on dist-test-slave-49h3"
server_key: "f526740ffe7b696c387d4e09b3423e9d"
server_key_iv: "b8ea23601aa75cefa058aaf6836c54dc"
server_key_version: "encryptionkey@0"
I20260808 13:11:30.812523 10495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:11:30.815950 10511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:30.817211 10495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:11:30.817376 10495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "a26075cf6a1b4067b4a5abdf2e4d869e"
format_stamp: "Formatted at 2026-08-08 13:11:30 on dist-test-slave-49h3"
server_key: "f526740ffe7b696c387d4e09b3423e9d"
server_key_iv: "b8ea23601aa75cefa058aaf6836c54dc"
server_key_version: "encryptionkey@0"
I20260808 13:11:30.817612 10495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:30.842484 10495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:30.846439 10495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:30.846715 10495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:30.864554 10495 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:33057
I20260808 13:11:30.864593 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:33057 every 8 connection(s)
I20260808 13:11:30.865931 10495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:11:30.869695 10565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:30.874567 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10495
I20260808 13:11:30.874818 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:11:30.875658 31982 external_mini_cluster.cc:1428] Setting key df0c5e25d451434612576423996814b7
I20260808 13:11:30.882229 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: Bootstrap starting.
I20260808 13:11:30.885340 10565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:30.886448 10565 log.cc:824] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:30.889532 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: No bootstrap required, opened a new log
Aug 08 13:11:30 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:30.897492 10565 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } }
I20260808 13:11:30.897883 10565 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:30.897993 10565 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26075cf6a1b4067b4a5abdf2e4d869e, State: Initialized, Role: FOLLOWER
I20260808 13:11:30.898357 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:30.877976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.31.59.190:33057)
0808 13:11:30.878829 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:30.878841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:30.878883 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:11:30.880795 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:30.880819 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:30.881212 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:30.881618 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:30.883161 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:30.884474 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:30.885890 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:30.886609 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:30.886805 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:30.890901 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:30.890909 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:30.890923 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:30.890964 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:30.894183 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:30.894876 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:30.894883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:30.894890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:30.895070 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:30.895441 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:30.895445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:30.895447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:30.896030 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:30.896379 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:30.896778 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 13:11:30.897198 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":375,"threads_started":1}
I20260808 13:11:30.898641 10565 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } }
I20260808 13:11:30.898890 10565 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:11:30.899031 10565 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:11:30.899197 10565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:30.901818 10565 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } }
I20260808 13:11:30.902309 10565 leader_election.cc:302] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26075cf6a1b4067b4a5abdf2e4d869e; no voters:
I20260808 13:11:30.902956 10565 leader_election.cc:288] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:11:30.903091 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:30.904062 10570 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [term 1 LEADER]: Becoming Leader. State: Replica: a26075cf6a1b4067b4a5abdf2e4d869e, State: Running, Role: LEADER
I20260808 13:11:30.904565 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } }
I20260808 13:11:30.905030 10565 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:11:30.907279 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } } }
I20260808 13:11:30.907688 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:30.909178 10579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:11:30.908950 10572 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26075cf6a1b4067b4a5abdf2e4d869e. Latest consensus state: current_term: 1 leader_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26075cf6a1b4067b4a5abdf2e4d869e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33057 } } }
I20260808 13:11:30.909309 10572 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e [sys.catalog]: This master's current role is: LEADER
I20260808 13:11:30.914348 10579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:11:30.923102 10579 catalog_manager.cc:1347] Generated new cluster ID: 9a0e3b010e1240cbad0c6d7f22088bd3
I20260808 13:11:30.923215 10579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:11:30.936260 10579 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:11:30.937579 10579 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:11:30.947391 10579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: Generated new TSK 0
I20260808 13:11:30.948218 10579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:11:31.015026 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:31.357959 10593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:31.358285 10593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:31.358430 10593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:31.365922 10593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:31.366077 10593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:31.366238 10593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:11:31.376379 10593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:31.377782 10593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:31.379102 10593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:31.392073 10598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:31.392010 10599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:31.392107 10603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:31.393375 10593 server_base.cc:1034] running on GCE node
I20260808 13:11:31.394032 10593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:31.394974 10593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:31.396196 10593 hybrid_clock.cc:648] HybridClock initialized: now 1786194691396145 us; error 36 us; skew 500 ppm
I20260808 13:11:31.396517 10593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:31 dist-test-slave-49h3 krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:31.402868 10593 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:11:31.404309 10593 webserver.cc:460] Webserver started at http://127.31.59.129:34257/ using document root <none> and password file <none>
I20260808 13:11:31.405014 10593 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:31.405131 10593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:31.405437 10593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:31.408510 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "89dd8942be72479d8b6321286fc607b9"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "131a60768a3d072c2fc117cb41ac9c6f"
server_key_iv: "ca83cf85c6467cf556cded792fe47641"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.409235 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "89dd8942be72479d8b6321286fc607b9"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "131a60768a3d072c2fc117cb41ac9c6f"
server_key_iv: "ca83cf85c6467cf556cded792fe47641"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.414258 10593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:11:31.417589 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:31.418819 10593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:11:31.418963 10593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "89dd8942be72479d8b6321286fc607b9"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "131a60768a3d072c2fc117cb41ac9c6f"
server_key_iv: "ca83cf85c6467cf556cded792fe47641"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.419134 10593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:31.436249 10593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:31.440511 10593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:31.440820 10593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:31.441818 10593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:31.443205 10593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:31.443300 10593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:31.443428 10593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:31.443482 10593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:31.473704 10593 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:40909
I20260808 13:11:31.473757 10727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:40909 every 8 connection(s)
I20260808 13:11:31.475005 10593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:11:31.484071 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10593
I20260808 13:11:31.484248 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:11:31.484905 31982 external_mini_cluster.cc:1428] Setting key 39304a5ca0172d0605eb3de16b86b645
Aug 08 13:11:31 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:31.495513 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:31.477483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:56453 (local address 127.31.59.190:33057)
0808 13:11:31.477925 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:31.477930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:31.478914 (+ 984us) server_negotiation.cc:407] Connection header received
0808 13:11:31.481038 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:31.481042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:31.481113 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:31.481199 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:31.482711 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:31.483448 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:31.485054 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:31.485670 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:31.485800 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:31.490073 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:31.490078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:31.490081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:31.490114 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:31.492547 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:31.493344 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:31.493348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:31.493351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:31.493489 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:31.493923 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:31.493925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:31.493927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:31.494173 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:31.494284 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:31.495078 (+ 794us) server_negotiation.cc:299] Negotiation successful
0808 13:11:31.495256 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":221,"threads_started":1}
I20260808 13:11:31.496881 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:31.478040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33057 (local address 127.31.59.129:56453)
0808 13:11:31.478764 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:31.478802 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:31.480773 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:31.481402 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:31.481411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:31.481991 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:31.482561 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:31.482579 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:31.483573 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:31.483577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:31.484893 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:31.484906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:31.486148 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:31.486154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:31.486379 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:31.487101 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:31.487134 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:31.489849 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:31.492681 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:31.492689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:31.492705 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:31.493245 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:31.493582 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:31.493586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:31.493589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:31.493835 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:31.494290 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:31.494297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:31.494717 (+ 420us) client_negotiation.cc:769] Sending connection context
0808 13:11:31.495008 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 13:11:31.495543 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":279,"threads_started":1}
I20260808 13:11:31.498440 10728 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33057
I20260808 13:11:31.498732 10728 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:31.499478 10728 heartbeater.cc:511] Master 127.31.59.190:33057 requested a full tablet report, sending...
I20260808 13:11:31.501567 10530 ts_manager.cc:194] Registered new tserver with Master: 89dd8942be72479d8b6321286fc607b9 (127.31.59.129:40909)
I20260808 13:11:31.503212 10530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:56453
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:11:31.547788 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:31.901609 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:31.901907 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:31.902057 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:31.909588 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:31.909708 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:31.909860 10734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:11:31.919917 10734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:31.921276 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:31.922719 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:31.935753 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:31.935998 10740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:31.935655 10742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:31.937557 10734 server_base.cc:1034] running on GCE node
I20260808 13:11:31.938390 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:31.939680 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:31.940901 10734 hybrid_clock.cc:648] HybridClock initialized: now 1786194691940834 us; error 50 us; skew 500 ppm
I20260808 13:11:31.941394 10734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:31 dist-test-slave-49h3 krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:31.944756 10734 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:11:31.946254 10734 webserver.cc:460] Webserver started at http://127.31.59.130:33079/ using document root <none> and password file <none>
I20260808 13:11:31.946946 10734 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:31.947057 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:31.947376 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:31.950448 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "91e56f38cbc84532a27bb4238449654e"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "424ee5955f39841f229f3de284c2b156"
server_key_iv: "bda3c1b6bc7c901a4a9d430fe92ca3c8"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.951169 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "91e56f38cbc84532a27bb4238449654e"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "424ee5955f39841f229f3de284c2b156"
server_key_iv: "bda3c1b6bc7c901a4a9d430fe92ca3c8"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.956362 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:11:31.959777 10749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:31.961076 10734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 13:11:31.961239 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "91e56f38cbc84532a27bb4238449654e"
format_stamp: "Formatted at 2026-08-08 13:11:31 on dist-test-slave-49h3"
server_key: "424ee5955f39841f229f3de284c2b156"
server_key_iv: "bda3c1b6bc7c901a4a9d430fe92ca3c8"
server_key_version: "encryptionkey@0"
I20260808 13:11:31.961421 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:31.982097 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:31.986577 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:31.986862 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:31.987874 10734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:31.989271 10734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:31.989367 10734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:31.989511 10734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:31.989571 10734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:32.020335 10734 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:46853
I20260808 13:11:32.020365 10871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:46853 every 8 connection(s)
I20260808 13:11:32.021795 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:11:32.024912 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10734
I20260808 13:11:32.025159 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:11:32.025797 31982 external_mini_cluster.cc:1428] Setting key 6864cfbf7513ae3508b517c8aee89b7c
Aug 08 13:11:32 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:32.042120 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.024501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:37075 (local address 127.31.59.190:33057)
0808 13:11:32.024951 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.024956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.026270 (+ 1314us) server_negotiation.cc:407] Connection header received
0808 13:11:32.028585 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.028590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.028670 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.028763 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:32.030338 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.031051 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.032576 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.032995 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.033105 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.036603 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:32.036608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:32.036612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:32.036645 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:32.039152 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.039862 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.039866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.039868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.039992 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.040405 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.040408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.040410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.040675 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:32.040808 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.041521 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.041765 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":235,"threads_started":1}
I20260808 13:11:32.043118 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.025233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33057 (local address 127.31.59.130:37075)
0808 13:11:32.026100 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.026146 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.028326 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.029017 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.029025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.029594 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:32.030170 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.030187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.031204 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.031207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.032435 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.032448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.033220 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.033228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.033468 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.034115 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:32.034141 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:32.036387 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:32.039283 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:32.039292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:32.039310 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:32.039741 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:32.040095 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:32.040100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:32.040104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:32.040291 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:32.040804 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:32.040811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:32.041130 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.041362 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.041941 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":347,"threads_started":1}
I20260808 13:11:32.044539 10872 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33057
I20260808 13:11:32.044864 10872 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:32.045826 10872 heartbeater.cc:511] Master 127.31.59.190:33057 requested a full tablet report, sending...
I20260808 13:11:32.047433 10530 ts_manager.cc:194] Registered new tserver with Master: 91e56f38cbc84532a27bb4238449654e (127.31.59.130:46853)
I20260808 13:11:32.048462 10530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:37075
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:11:32.086699 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:11:32.412899 10878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:11:32.413163 10878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:11:32.413296 10878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:11:32.420351 10878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:11:32.420471 10878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:11:32.420614 10878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:11:32.430337 10878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:11:32.431649 10878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:11:32.432964 10878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:11:32.445613 10883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:32.445662 10886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:11:32.445580 10884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:11:32.446266 10878 server_base.cc:1034] running on GCE node
I20260808 13:11:32.446882 10878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:11:32.447794 10878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:11:32.448993 10878 hybrid_clock.cc:648] HybridClock initialized: now 1786194692448941 us; error 47 us; skew 500 ppm
I20260808 13:11:32.449288 10878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:11:32 dist-test-slave-49h3 krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194692, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:11:32.452288 10878 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:11:32.453715 10878 webserver.cc:460] Webserver started at http://127.31.59.131:35423/ using document root <none> and password file <none>
I20260808 13:11:32.454396 10878 fs_manager.cc:359] Metadata directory not provided
I20260808 13:11:32.454509 10878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:11:32.454797 10878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:11:32.457690 10878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "92a7077575d74ccb9648e7c382d9da78"
format_stamp: "Formatted at 2026-08-08 13:11:32 on dist-test-slave-49h3"
server_key: "9134773843fcfcb61ae33256730c59f1"
server_key_iv: "6f4462eee2f55715a2d60bce32372d8f"
server_key_version: "encryptionkey@0"
I20260808 13:11:32.458446 10878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "92a7077575d74ccb9648e7c382d9da78"
format_stamp: "Formatted at 2026-08-08 13:11:32 on dist-test-slave-49h3"
server_key: "9134773843fcfcb61ae33256730c59f1"
server_key_iv: "6f4462eee2f55715a2d60bce32372d8f"
server_key_version: "encryptionkey@0"
I20260808 13:11:32.463542 10878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:11:32.466938 10893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:32.468179 10878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 13:11:32.468333 10878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "92a7077575d74ccb9648e7c382d9da78"
format_stamp: "Formatted at 2026-08-08 13:11:32 on dist-test-slave-49h3"
server_key: "9134773843fcfcb61ae33256730c59f1"
server_key_iv: "6f4462eee2f55715a2d60bce32372d8f"
server_key_version: "encryptionkey@0"
I20260808 13:11:32.468509 10878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:11:32.487946 10878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:11:32.491632 10878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:11:32.491931 10878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:11:32.492847 10878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:11:32.494174 10878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:11:32.494253 10878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:32.494369 10878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:11:32.494421 10878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:11:32.506670 10728 heartbeater.cc:503] Master 127.31.59.190:33057 was elected leader, sending a full tablet report...
I20260808 13:11:32.524873 10878 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:40687
I20260808 13:11:32.524923 11009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:40687 every 8 connection(s)
I20260808 13:11:32.526186 10878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:11:32.532812 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 10878
I20260808 13:11:32.532984 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:11:32.533433 31982 external_mini_cluster.cc:1428] Setting key bb1e5d1269d6d69c30c9187c592673db
Aug 08 13:11:32 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194692, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:11:32.546350 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.528555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:35507 (local address 127.31.59.190:33057)
0808 13:11:32.528718 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.528722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.529856 (+ 1134us) server_negotiation.cc:407] Connection header received
0808 13:11:32.531750 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.531754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.531837 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.531922 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:32.533675 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.534696 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.536520 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.536924 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.537037 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.540453 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:32.540458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:32.540461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:32.540497 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:32.543067 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.543939 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.543944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.543946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.544080 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.544504 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.544507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.544509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.544809 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:32.544957 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.545758 (+ 801us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.545998 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:11:32.547426 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.528967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33057 (local address 127.31.59.131:35507)
0808 13:11:32.529686 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.529734 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.531495 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.532102 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.532110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.532779 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:11:32.533478 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.533502 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.534857 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.534861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.536390 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.536401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.537198 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.537207 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.537407 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.538053 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:11:32.538089 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:11:32.540240 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:11:32.543311 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:32.543322 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:32.543339 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:32.543800 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:32.544189 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:11:32.544194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:11:32.544198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:11:32.544388 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:11:32.544940 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:11:32.544948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:11:32.545341 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.545646 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.546239 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":264,"threads_started":1}
I20260808 13:11:32.548832 11010 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33057
I20260808 13:11:32.549134 11010 heartbeater.cc:464] Registering TS with master...
I20260808 13:11:32.549863 11010 heartbeater.cc:511] Master 127.31.59.190:33057 requested a full tablet report, sending...
I20260808 13:11:32.551153 10530 ts_manager.cc:194] Registered new tserver with Master: 92a7077575d74ccb9648e7c382d9da78 (127.31.59.131:40687)
I20260808 13:11:32.551926 10530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:35507
I20260808 13:11:32.560977 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:11:32.577684 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.565287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.31.59.190:33057)
0808 13:11:32.565436 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.565440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.566027 (+ 587us) server_negotiation.cc:407] Connection header received
0808 13:11:32.566218 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.566223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.566320 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.566399 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:32.567722 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.568447 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.569682 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.570098 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.570211 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.572487 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:32.572492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:32.572495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:32.572527 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:32.575091 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.575701 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.575707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.575711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.575854 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.576204 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.576208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.576212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.576535 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:32.576783 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.577121 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.577349 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:11:32.581666 10530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:11:32.588745 10530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:11:32.611893 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.601201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.31.59.130:46853)
0808 13:11:32.604079 (+ 2878us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.604089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.604120 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:11:32.604227 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.604234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.604556 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.605569 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:11:32.605847 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.607104 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.609637 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.610766 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.610891 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.611101 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.611409 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.611574 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2695,"thread_start_us":2551,"threads_started":1}
I20260808 13:11:32.611912 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.600671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:46853 (local address 127.0.0.1:52854)
0808 13:11:32.602814 (+ 2143us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.602834 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.602960 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.604822 (+ 1862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.604830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.604862 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:11:32.605317 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.605330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.607310 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.607316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.609483 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.609501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.610973 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.610979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.611189 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.611227 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.611332 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.611521 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1999,"thread_start_us":1858,"threads_started":1}
I20260808 13:11:32.613706 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.602895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40909 (local address 127.0.0.1:46504)
0808 13:11:32.603424 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.603440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.603531 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.605347 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.605352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.605382 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:11:32.605774 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.605784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.607609 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.607614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.609981 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.609999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.612093 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.612100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.612284 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.612318 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.613211 (+ 893us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.613349 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":292,"threads_started":1}
I20260808 13:11:32.613775 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.603243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46504 (local address 127.31.59.129:40909)
0808 13:11:32.604433 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.604441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.604470 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:11:32.604562 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.604569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.604839 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.605065 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:11:32.605936 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.607487 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.610136 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.611620 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.611738 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.611944 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.613265 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.613421 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":1946,"threads_started":1}
I20260808 13:11:32.614526 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.600550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:40687 (local address 127.0.0.1:38040)
0808 13:11:32.601348 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.601388 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.601586 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.606993 (+ 5407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.606998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.607023 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:11:32.607451 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.607462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.609505 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.609510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.611384 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.611400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.613815 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.613822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.613985 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.614015 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.614105 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.614226 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":1277,"threads_started":1}
I20260808 13:11:32.615259 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.600728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.31.59.131:40687)
0808 13:11:32.606436 (+ 5708us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.606444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.606475 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:11:32.606570 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.606577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.606768 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.606943 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:11:32.607693 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.609338 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.611567 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.613300 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.613423 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.613656 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.614813 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.614964 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5539,"thread_start_us":5381,"threads_started":1}
I20260808 13:11:32.616235 10799 tablet_service.cc:1467] Processing CreateTablet for tablet c2d326d497294e309215028f012041ee (DEFAULT_TABLE table=test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:32.616950 10659 tablet_service.cc:1467] Processing CreateTablet for tablet c2d326d497294e309215028f012041ee (DEFAULT_TABLE table=test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:32.617775 10799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d326d497294e309215028f012041ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:32.618358 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d326d497294e309215028f012041ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:32.618553 10941 tablet_service.cc:1467] Processing CreateTablet for tablet c2d326d497294e309215028f012041ee (DEFAULT_TABLE table=test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:11:32.619867 10941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d326d497294e309215028f012041ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:11:32.628943 11028 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Bootstrap starting.
I20260808 13:11:32.632409 11029 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Bootstrap starting.
I20260808 13:11:32.633070 11028 tablet_bootstrap.cc:654] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:32.633867 11030 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Bootstrap starting.
I20260808 13:11:32.634436 11028 log.cc:824] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:32.635782 11029 tablet_bootstrap.cc:654] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:32.636716 11029 log.cc:824] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:32.637141 11030 tablet_bootstrap.cc:654] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Neither blocks nor log segments found. Creating new log.
I20260808 13:11:32.637770 11028 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: No bootstrap required, opened a new log
I20260808 13:11:32.638058 11028 ts_tablet_manager.cc:1397] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 13:11:32.638450 11030 log.cc:824] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Log is configured to *not* fsync() on all Append() calls
I20260808 13:11:32.639412 11029 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: No bootstrap required, opened a new log
I20260808 13:11:32.640623 11029 ts_tablet_manager.cc:1397] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:11:32.642290 11030 tablet_bootstrap.cc:492] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: No bootstrap required, opened a new log
I20260808 13:11:32.642537 11030 ts_tablet_manager.cc:1397] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:11:32.647543 11029 raft_consensus.cc:348] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.647907 11029 raft_consensus.cc:374] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:32.648017 11029 raft_consensus.cc:729] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dd8942be72479d8b6321286fc607b9, State: Initialized, Role: FOLLOWER
I20260808 13:11:32.648104 11028 raft_consensus.cc:348] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.648521 11028 raft_consensus.cc:374] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:32.648618 11028 raft_consensus.cc:729] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e56f38cbc84532a27bb4238449654e, State: Initialized, Role: FOLLOWER
I20260808 13:11:32.648623 11029 consensus_queue.cc:260] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.649166 11028 consensus_queue.cc:260] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.652045 11029 ts_tablet_manager.cc:1428] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260808 13:11:32.652870 11030 raft_consensus.cc:348] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.653328 11030 raft_consensus.cc:374] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:11:32.653543 11030 raft_consensus.cc:729] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a7077575d74ccb9648e7c382d9da78, State: Initialized, Role: FOLLOWER
I20260808 13:11:32.654186 10872 heartbeater.cc:503] Master 127.31.59.190:33057 was elected leader, sending a full tablet report...
I20260808 13:11:32.654831 11028 ts_tablet_manager.cc:1428] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260808 13:11:32.655313 11030 consensus_queue.cc:260] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.656445 11010 heartbeater.cc:503] Master 127.31.59.190:33057 was elected leader, sending a full tablet report...
I20260808 13:11:32.657092 11030 ts_tablet_manager.cc:1428] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 13:11:32.661393 11037 raft_consensus.cc:482] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:11:32.661672 11037 raft_consensus.cc:504] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.663563 11037 leader_election.cc:288] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a7077575d74ccb9648e7c382d9da78 (127.31.59.131:40687), 89dd8942be72479d8b6321286fc607b9 (127.31.59.129:40909)
I20260808 13:11:32.670588 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.664158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:53061 (local address 127.31.59.129:40909)
0808 13:11:32.664380 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.664386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.665015 (+ 629us) server_negotiation.cc:407] Connection header received
0808 13:11:32.665166 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.665171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.665242 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.665328 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:11:32.666006 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.666705 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.668523 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.669566 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.669712 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.669783 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.670222 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.670353 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:11:32.670630 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.663911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:55683 (local address 127.31.59.131:40687)
0808 13:11:32.664094 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.664100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.664816 (+ 716us) server_negotiation.cc:407] Connection header received
0808 13:11:32.664995 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.665001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.665068 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.665155 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:11:32.665831 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.666625 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.668511 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.669541 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.669703 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.669779 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.670258 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.670383 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:11:32.670761 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.664013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40909 (local address 127.31.59.130:53061)
0808 13:11:32.664906 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.664920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.665045 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.665341 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.665344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.665372 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:11:32.665838 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.665850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.666842 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.666848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.668372 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.668385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.669822 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.669830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.670074 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.670100 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.670180 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.670327 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":5,"thread_start_us":314,"threads_started":1}
I20260808 13:11:32.671029 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.663907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:40687 (local address 127.31.59.130:55683)
0808 13:11:32.664687 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0808 13:11:32.664715 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:11:32.664868 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:11:32.665176 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:11:32.665180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:11:32.665227 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:11:32.665688 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.665699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.666747 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.666753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.668366 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:11:32.668378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.669931 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.669938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:11:32.670097 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.670130 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:11:32.670211 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 13:11:32.670362 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":62,"thread_start_us":714,"threads_started":1}
I20260808 13:11:32.671762 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d326d497294e309215028f012041ee" candidate_uuid: "91e56f38cbc84532a27bb4238449654e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89dd8942be72479d8b6321286fc607b9" is_pre_election: true
I20260808 13:11:32.671911 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d326d497294e309215028f012041ee" candidate_uuid: "91e56f38cbc84532a27bb4238449654e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a7077575d74ccb9648e7c382d9da78" is_pre_election: true
I20260808 13:11:32.672284 10679 raft_consensus.cc:2454] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e56f38cbc84532a27bb4238449654e in term 0.
I20260808 13:11:32.672442 10963 raft_consensus.cc:2454] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e56f38cbc84532a27bb4238449654e in term 0.
I20260808 13:11:32.672904 10751 leader_election.cc:302] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89dd8942be72479d8b6321286fc607b9, 91e56f38cbc84532a27bb4238449654e; no voters:
I20260808 13:11:32.673334 11037 raft_consensus.cc:2781] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:11:32.673471 11037 raft_consensus.cc:482] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:11:32.673580 11037 raft_consensus.cc:3037] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:32.675920 11037 raft_consensus.cc:504] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.676484 11037 leader_election.cc:288] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [CANDIDATE]: Term 1 election: Requested vote from peers 92a7077575d74ccb9648e7c382d9da78 (127.31.59.131:40687), 89dd8942be72479d8b6321286fc607b9 (127.31.59.129:40909)
I20260808 13:11:32.676918 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d326d497294e309215028f012041ee" candidate_uuid: "91e56f38cbc84532a27bb4238449654e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a7077575d74ccb9648e7c382d9da78"
I20260808 13:11:32.676923 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d326d497294e309215028f012041ee" candidate_uuid: "91e56f38cbc84532a27bb4238449654e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89dd8942be72479d8b6321286fc607b9"
I20260808 13:11:32.677089 10679 raft_consensus.cc:3037] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:32.677098 10963 raft_consensus.cc:3037] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:11:32.679509 10679 raft_consensus.cc:2454] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e56f38cbc84532a27bb4238449654e in term 1.
I20260808 13:11:32.679580 10963 raft_consensus.cc:2454] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e56f38cbc84532a27bb4238449654e in term 1.
I20260808 13:11:32.679986 10753 leader_election.cc:302] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e56f38cbc84532a27bb4238449654e, 92a7077575d74ccb9648e7c382d9da78; no voters:
I20260808 13:11:32.680259 11037 raft_consensus.cc:2781] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:11:32.680739 11037 raft_consensus.cc:686] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 1 LEADER]: Becoming Leader. State: Replica: 91e56f38cbc84532a27bb4238449654e, State: Running, Role: LEADER
I20260808 13:11:32.681249 11037 consensus_queue.cc:237] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } }
I20260808 13:11:32.685818 10530 catalog_manager.cc:5495] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e56f38cbc84532a27bb4238449654e (127.31.59.130). New cstate: current_term: 1 leader_uuid: "91e56f38cbc84532a27bb4238449654e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e56f38cbc84532a27bb4238449654e" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 46853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20260808 13:11:32.710640 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.705357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52868 (local address 127.31.59.130:46853)
0808 13:11:32.705593 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.705599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.705618 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:11:32.705707 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.705710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.705799 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.705927 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:11:32.706374 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.707072 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.708452 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.708874 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.708981 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.709519 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:11:32.710044 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:11:32.710197 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.710300 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.710384 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:11:32.719022 10963 raft_consensus.cc:1261] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 1 FOLLOWER]: Refusing update from remote peer 91e56f38cbc84532a27bb4238449654e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:32.720036 11037 consensus_queue.cc:1035] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a7077575d74ccb9648e7c382d9da78" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:11:32.720351 10679 raft_consensus.cc:1261] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 1 FOLLOWER]: Refusing update from remote peer 91e56f38cbc84532a27bb4238449654e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:11:32.721154 11037 consensus_queue.cc:1035] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [LEADER]: Connected to new peer: Peer: permanent_uuid: "89dd8942be72479d8b6321286fc607b9" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:11:32.727285 10729 tablet.cc:2369] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:11:32.734367 11044 mvcc.cc:205] Tried to move back new op lower bound from 7316253461365395456 to 7316253461226020864. Current Snapshot: MvccSnapshot[applied={T|T < 7316253461365395456}]
I20260808 13:11:32.734906 11046 mvcc.cc:205] Tried to move back new op lower bound from 7316253461365395456 to 7316253461226020864. Current Snapshot: MvccSnapshot[applied={T|T < 7316253461365395456}]
I20260808 13:11:32.749611 10530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:32.749893 10530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:11:32.753479 10530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a26075cf6a1b4067b4a5abdf2e4d869e: Sending DeleteTablet for 3 replicas of tablet c2d326d497294e309215028f012041ee
I20260808 13:11:32.754786 10941 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d326d497294e309215028f012041ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:32 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:38040
I20260808 13:11:32.754874 10799 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d326d497294e309215028f012041ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:32 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:52854
I20260808 13:11:32.755116 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d326d497294e309215028f012041ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:11:32 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46504
I20260808 13:11:32.756695 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10593
I20260808 13:11:32.767233 11055 tablet_replica.cc:330] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: stopping tablet replica
I20260808 13:11:32.767781 11055 raft_consensus.cc:2229] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:32.768352 11055 raft_consensus.cc:2258] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:32.771389 11056 tablet_replica.cc:330] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: stopping tablet replica
I20260808 13:11:32.771952 11054 tablet_replica.cc:330] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: stopping tablet replica
I20260808 13:11:32.771960 11056 raft_consensus.cc:2229] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:11:32.772370 11054 raft_consensus.cc:2229] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:11:32.772785 11056 raft_consensus.cc:2258] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:32.773536 11055 ts_tablet_manager.cc:1905] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:32.774816 11056 ts_tablet_manager.cc:1905] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:32.775372 11054 raft_consensus.cc:2258] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:11:32.777575 11054 ts_tablet_manager.cc:1905] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:11:32.780701 11055 ts_tablet_manager.cc:1918] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:32.780879 11055 log.cc:1196] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/c2d326d497294e309215028f012041ee
I20260808 13:11:32.781433 11055 ts_tablet_manager.cc:1939] T c2d326d497294e309215028f012041ee P 92a7077575d74ccb9648e7c382d9da78: Deleting consensus metadata
I20260808 13:11:32.783429 10514 catalog_manager.cc:4841] TS 92a7077575d74ccb9648e7c382d9da78 (127.31.59.131:40687): tablet c2d326d497294e309215028f012041ee (table test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]) successfully deleted
I20260808 13:11:32.784375 11056 ts_tablet_manager.cc:1918] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:32.784601 11056 log.cc:1196] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/c2d326d497294e309215028f012041ee
I20260808 13:11:32.785200 11056 ts_tablet_manager.cc:1939] T c2d326d497294e309215028f012041ee P 89dd8942be72479d8b6321286fc607b9: Deleting consensus metadata
I20260808 13:11:32.785358 11054 ts_tablet_manager.cc:1918] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:11:32.785567 11054 log.cc:1196] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/c2d326d497294e309215028f012041ee
I20260808 13:11:32.786154 11054 ts_tablet_manager.cc:1939] T c2d326d497294e309215028f012041ee P 91e56f38cbc84532a27bb4238449654e: Deleting consensus metadata
Aug 08 13:11:32 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:11:32.787437 10513 catalog_manager.cc:4841] TS 89dd8942be72479d8b6321286fc607b9 (127.31.59.129:40909): tablet c2d326d497294e309215028f012041ee (table test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]) successfully deleted
I20260808 13:11:32.787737 10513 catalog_manager.cc:4841] TS 91e56f38cbc84532a27bb4238449654e (127.31.59.130:46853): tablet c2d326d497294e309215028f012041ee (table test-table [id=720fe710f8d84ee0ad2bf1e54c66d1c5]) successfully deleted
I20260808 13:11:32.795310 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.758453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.31.59.129:40909)
0808 13:11:32.758647 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.758654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.770624 (+ 11970us) server_negotiation.cc:407] Connection header received
0808 13:11:32.770842 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.770848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.770948 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.771056 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:32.773885 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.775000 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.776902 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.777538 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.777642 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.788183 (+ 10541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:32.788194 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:32.788209 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:32.788273 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:32.792474 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.793327 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.793334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.793336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.793510 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.793909 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.793912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.793914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.794281 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:32.794504 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.794855 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.795021 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:11:32.795957 10722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:32.909142 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10593
I20260808 13:11:32.929083 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10734
Aug 08 13:11:32 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:11:32.944779 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:32.929892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.31.59.130:46853)
0808 13:11:32.930114 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:32.930120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:32.930139 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:11:32.930283 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:32.930287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:32.930360 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:32.930443 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:32.931666 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.932608 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.933916 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:32.934354 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:32.934467 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:32.938755 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:32.938769 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:32.938787 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:32.938826 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:32.942129 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.942775 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.942783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.942785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.942923 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:32.943279 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:32.943283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:32.943285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:32.943653 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:32.943990 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:32.944212 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 13:11:32.944425 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:11:32.945566 10867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:33.061748 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10734
I20260808 13:11:33.083299 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10878
Aug 08 13:11:33 dist-test-slave-49h3 krb5kdc[10481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:11:33.096976 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:11:33.084002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.31.59.131:40687)
0808 13:11:33.084158 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 13:11:33.084162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:11:33.084196 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:11:33.084351 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:11:33.084355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:11:33.084421 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:11:33.084502 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:11:33.085735 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:33.086493 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:33.087606 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:11:33.087984 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:11:33.088069 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:11:33.092236 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:11:33.092252 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:11:33.092268 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:11:33.092301 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:11:33.094813 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:33.095385 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:33.095391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:33.095393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:33.095516 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:11:33.095776 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:11:33.095779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:11:33.095781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:11:33.096132 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:11:33.096343 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 13:11:33.096555 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 13:11:33.096715 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:11:33.097654 11005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:33.186129 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10878
I20260808 13:11:33.203635 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 10495
I20260808 13:11:33.204792 10560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:11:33.316591 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 10495
2026-08-08T13:11:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4250 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:11:33 dist-test-slave-49h3 krb5kdc[11066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:11:33 dist-test-slave-49h3 krb5kdc[11066](info): set up 2 sockets
Aug 08 13:11:33 dist-test-slave-49h3 krb5kdc[11066](info): commencing operation
krb5kdc: starting...
W20260808 13:11:34.401136 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:11:34 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:11:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangeradmin_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangeradmin_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangerlookup_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangerlookup_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
I20260808 13:11:34.796978 31982 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskePcvnh/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/postgres -l logfile start
2026-08-08 13:11:37.079 UTC [11096] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 13:11:37.079 UTC [11096] LOG: listening on IPv4 address "127.31.59.148", port 36183
2026-08-08 13:11:37.082 UTC [11096] LOG: listening on Unix socket "/tmp/.s.PGSQL.36183"
2026-08-08 13:11:37.104 UTC [11098] LOG: database system was shut down at 2026-08-08 13:11:35 UTC
2026-08-08 13:11:37.108 UTC [11096] LOG: database system is ready to accept connections
I20260808 13:11:38.052326 31982 mini_postgres.cc:96] Postgres bound to 36183
2026-08-08 13:11:38.076 UTC [11107] FATAL: database "slave" does not exist
127.31.59.148:36183 - accepting connections
I20260808 13:11:38.077096 31982 mini_ranger.cc:161] Starting Ranger...
I20260808 13:11:38.108047 31982 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 13:11:38.548959 31982 mini_ranger.cc:87] Created ranger Postgres database
I20260808 13:11:38.549129 31982 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin
2026-08-08 13:11:38,857 [I] DB FLAVOR :POSTGRES
2026-08-08 13:11:38,857 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:11:38,857 [I] Checking connection..
2026-08-08 13:11:38,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 13:11:39,124 [I] Checking connection passed.
2026-08-08 13:11:39,124 [I] --------- Verifying version history table ---------
2026-08-08 13:11:39,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:11:39,386 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:11:39,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:11:39,647 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:11:39,648 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 13:11:39,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 13:11:39.880 UTC [11178] WARNING: there is no transaction in progress
2026-08-08 13:11:39,887 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 13:11:39,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:11:40,131 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 13:11:40,131 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:11:40,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 13:11:40,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 13:11:40,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-08 13:11:40,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:11:41,111 [I] Table x_portal_user does not exist in database ranger
2026-08-08 13:11:41,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:11:41,348 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 13:11:41,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:11:41,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:11:41,850 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 13:11:41,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 13:11:42.749 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.766 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.781 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.796 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.814 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.830 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.883 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.889 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.903 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.912 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.922 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.934 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.943 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.949 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.955 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:42.961 UTC [11288] WARNING: there is no transaction in progress
2026-08-08 13:11:43,542 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 13:11:43,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 13:11:43,796 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 13:11:43,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:11:44,049 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 13:11:44,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:11:44,289 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 13:11:44,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:11:44,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:11:44,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-08 13:11:45,031 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 13:11:45,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 13:11:45,288 [I] DB_PATCHES have already been applied
I20260808 13:11:45.294983 31982 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskePcvnh/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 13:11:45.295115 31982 mini_ranger.cc:193] Using host: 127.31.59.148
I20260808 13:11:45.316404 31982 mini_ranger.cc:227] Ranger admin URL: http://127.31.59.148:47503
Aug 08, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 1:11:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 1:11:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 13:11:46.429579 31982 mini_ranger.cc:160] Time spent starting Ranger: real 8.352s user 0.000s sys 0.068s
Aug 08, 2026 1:12:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 1:12:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 1:12:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 1:12:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 1:12:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 13:12:08.462445 31982 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangerkms_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/rangerkms_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.59.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 13:12:08 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194728, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 13:12:08.657823 31982 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 13:12:08.690351 31982 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 13:12:09.436638 31982 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 13:12:09.436779 31982 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms
I20260808 13:12:09.436926 31982 mini_ranger_kms.cc:227] Using postgres at 127.31.59.148:36183
2026-08-08 13:12:09,950 [I] DB FLAVOR :POSTGRES
2026-08-08 13:12:09,951 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:12:09,951 [I] Checking connection
2026-08-08 13:12:09,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 13:12:10,211 [I] connection success
2026-08-08 13:12:10,212 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 13:12:10,212 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 13:12:10,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 13:12:10,485 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 13:12:10,486 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:12:10,486 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 13:12:10,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:36183/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 13:12:10,761 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 13:12:11.229503 31982 mini_ranger_kms.cc:310] Created kms service
I20260808 13:12:11.421993 31982 mini_ranger_kms.cc:326] Created kudu user
I20260808 13:12:11.465467 31982 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 13:12:11 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194728, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:11.856813 31982 mini_ranger_kms.cc:384] Added ranger policy
I20260808 13:12:11.858225 31982 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms:/tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 13:12:11.858284 31982 mini_ranger_kms.cc:240] Using host: 127.31.59.148
I20260808 13:12:11.883898 31982 mini_ranger_kms.cc:276] Ranger KMS PID: 11481
I20260808 13:12:11.884106 31982 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.59.148:42155
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 1:12:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 13:12:14.120886 31982 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.463s user 0.140s sys 0.415s
I20260808 13:12:14.121294 31982 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 13:12:14.121567 31982 mini_ranger_kms.cc:401] 127.31.59.148:42155/kms/v1/keys
Aug 08 13:12:14 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 13:12:14 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
Aug 08 13:12:15 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
Aug 08 13:12:17 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:12:17.452608 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36841
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:36841
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:42155/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:12:18.100381 11536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:12:18.100792 11536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:12:18.101016 11536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:12:18.112102 11536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:12:18.112340 11536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:12:18.112457 11536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:12:18.112600 11536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:12:18.112777 11536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:12:18.127774 11536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.59.148:42155/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:36841
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:12:18.129355 11536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:12:18.130909 11536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:12:18.148134 11542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:18.148038 11545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:18.148272 11543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:12:18.149760 11536 server_base.cc:1034] running on GCE node
I20260808 13:12:18.150579 11536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:12:18.152216 11536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:12:18.153529 11536 hybrid_clock.cc:648] HybridClock initialized: now 1786194738153381 us; error 40 us; skew 500 ppm
I20260808 13:12:18.153965 11536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:12:18 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194738, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:12:18.163552 11536 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:12:18.165668 11536 webserver.cc:460] Webserver started at http://127.31.59.190:40059/ using document root <none> and password file <none>
I20260808 13:12:18.166445 11536 fs_manager.cc:359] Metadata directory not provided
I20260808 13:12:18.166579 11536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:12:18.166996 11536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:12:18 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194738, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:18.241477 11536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "6140b16e0706443e823f9bc7a1da45dc"
format_stamp: "Formatted at 2026-08-08 13:12:18 on dist-test-slave-49h3"
server_key: "d2f3fa0264015658b2bae2250305c5fc"
server_key_iv: "c2c328b6a63e021570e23b6ff6c3acd7"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:18.242554 11536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "6140b16e0706443e823f9bc7a1da45dc"
format_stamp: "Formatted at 2026-08-08 13:12:18 on dist-test-slave-49h3"
server_key: "d2f3fa0264015658b2bae2250305c5fc"
server_key_iv: "c2c328b6a63e021570e23b6ff6c3acd7"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:18.249791 11536 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260808 13:12:18.271339 11554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:18.273362 11536 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260808 13:12:18.273679 11536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "6140b16e0706443e823f9bc7a1da45dc"
format_stamp: "Formatted at 2026-08-08 13:12:18 on dist-test-slave-49h3"
server_key: "d2f3fa0264015658b2bae2250305c5fc"
server_key_iv: "c2c328b6a63e021570e23b6ff6c3acd7"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:18.274099 11536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:12:18.318610 11536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:12:18.325662 11536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:12:18.326256 11536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:12:18.371516 11536 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:36841
I20260808 13:12:18.371630 11608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:36841 every 8 connection(s)
I20260808 13:12:18.373654 11536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:12:18.379580 11609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:12:18.380890 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 11536
I20260808 13:12:18.381158 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:12:18.382272 31982 external_mini_cluster.cc:1428] Setting key f8d9d0284e2b7c729890c80f292fefd6
I20260808 13:12:18.404165 11609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: Bootstrap starting.
Aug 08 13:12:18 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:12:18.410101 11609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: Neither blocks nor log segments found. Creating new log.
I20260808 13:12:18.411600 11609 log.cc:824] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: Log is configured to *not* fsync() on all Append() calls
I20260808 13:12:18.415743 11609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: No bootstrap required, opened a new log
I20260808 13:12:18.420351 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:18.385084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.31.59.190:36841)
0808 13:12:18.387179 (+ 2095us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:18.387195 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:18.387262 (+ 67us) server_negotiation.cc:407] Connection header received
0808 13:12:18.391857 (+ 4595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:18.391891 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:18.392553 (+ 662us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:18.394991 (+ 2438us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:18.395753 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:18.398801 (+ 3048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:18.401003 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:18.402608 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:18.403478 (+ 870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:18.409991 (+ 6513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:18.410009 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:18.410034 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:18.410128 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:18.415138 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:18.416066 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:18.416074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:18.416081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:18.416326 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:18.416804 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:18.416809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:18.416812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:18.417664 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:18.418034 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:18.418421 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 13:12:18.418824 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1740,"thread_start_us":1239,"threads_started":1}
I20260808 13:12:18.428645 11609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } }
I20260808 13:12:18.429294 11609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:12:18.429563 11609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6140b16e0706443e823f9bc7a1da45dc, State: Initialized, Role: FOLLOWER
I20260808 13:12:18.430384 11609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } }
I20260808 13:12:18.430917 11609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:12:18.431133 11609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:12:18.431365 11609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:12:18.434805 11609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } }
I20260808 13:12:18.435369 11609 leader_election.cc:302] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6140b16e0706443e823f9bc7a1da45dc; no voters:
I20260808 13:12:18.436478 11609 leader_election.cc:288] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:12:18.436571 11614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:12:18.438396 11614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [term 1 LEADER]: Becoming Leader. State: Replica: 6140b16e0706443e823f9bc7a1da45dc, State: Running, Role: LEADER
I20260808 13:12:18.439306 11614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } }
I20260808 13:12:18.439895 11609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:12:18.443274 11615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6140b16e0706443e823f9bc7a1da45dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } } }
I20260808 13:12:18.443087 11616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6140b16e0706443e823f9bc7a1da45dc. Latest consensus state: current_term: 1 leader_uuid: "6140b16e0706443e823f9bc7a1da45dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6140b16e0706443e823f9bc7a1da45dc" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36841 } } }
I20260808 13:12:18.444321 11615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [sys.catalog]: This master's current role is: LEADER
I20260808 13:12:18.444367 11616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc [sys.catalog]: This master's current role is: LEADER
I20260808 13:12:18.449662 11621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:12:18.458014 11621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:12:18.462860 11609 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 13:12:18.488912 11621 catalog_manager.cc:1347] Generated new cluster ID: ba4635de44ee4612afd89f26d1277221
I20260808 13:12:18.489075 11621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:12:18.522250 11621 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:12:18.524751 11621 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:12:18.535032 11621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: Generated new TSK 0
I20260808 13:12:18.536055 11621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:12:19.351212 11654 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:12:19.461494 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:42155/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:12:19 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194739, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 13:12:20.141135 11657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:12:20.141746 11657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:12:20.142001 11657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:12:20.157224 11657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:12:20.157541 11657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:12:20.157838 11657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:12:20.177037 11657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.59.148:42155/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:12:20.179185 11657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:12:20.181221 11657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:12:20.215493 11668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:20.216871 11667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:20.217129 11670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:12:20.224491 11657 server_base.cc:1034] running on GCE node
I20260808 13:12:20.225538 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:12:20.227161 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:12:20.228397 11657 hybrid_clock.cc:648] HybridClock initialized: now 1786194740228296 us; error 94 us; skew 500 ppm
I20260808 13:12:20.228852 11657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:12:20 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194740, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:12:20.234949 11657 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:12:20.239991 11657 webserver.cc:460] Webserver started at http://127.31.59.129:45153/ using document root <none> and password file <none>
I20260808 13:12:20.241400 11657 fs_manager.cc:359] Metadata directory not provided
I20260808 13:12:20.241768 11657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:12:20.242400 11657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:12:20 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194740, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:20.269407 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "ddef6e04ac6a4c24b709b85bec72f2f2"
format_stamp: "Formatted at 2026-08-08 13:12:20 on dist-test-slave-49h3"
server_key: "b066f30b1e3cbd33e840a520ea14ec53"
server_key_iv: "60ef98c2017e71302408abbd1aa9e8d5"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:20.271744 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "ddef6e04ac6a4c24b709b85bec72f2f2"
format_stamp: "Formatted at 2026-08-08 13:12:20 on dist-test-slave-49h3"
server_key: "b066f30b1e3cbd33e840a520ea14ec53"
server_key_iv: "60ef98c2017e71302408abbd1aa9e8d5"
server_key_version: "kuduclusterkey@0"
Aug 08 13:12:20 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194739, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:20.287607 11657 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.008s sys 0.004s
I20260808 13:12:20.328679 11679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:20.330765 11657 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260808 13:12:20.331064 11657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "ddef6e04ac6a4c24b709b85bec72f2f2"
format_stamp: "Formatted at 2026-08-08 13:12:20 on dist-test-slave-49h3"
server_key: "b066f30b1e3cbd33e840a520ea14ec53"
server_key_iv: "60ef98c2017e71302408abbd1aa9e8d5"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:20.331442 11657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:12:20.380445 11657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:12:20.387806 11657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:12:20.388477 11657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:12:20.396528 11657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:12:20.405833 11657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:12:20.406136 11657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:20.406459 11657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:12:20.406667 11657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:20.570641 11657 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45403
I20260808 13:12:20.570787 11799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45403 every 8 connection(s)
I20260808 13:12:20.572618 11657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:12:20.582926 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 11657
I20260808 13:12:20.583169 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:12:20.584151 31982 external_mini_cluster.cc:1428] Setting key 9a4cd92134169719c26a8f0ac03ec679
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Aug 08 13:12:20 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194740, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:12:20.647872 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:20.576312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:42821 (local address 127.31.59.190:36841)
0808 13:12:20.607287 (+ 30975us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:20.607294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:20.607323 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:12:20.611196 (+ 3873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:20.611204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:20.611319 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:20.611458 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:20.620274 (+ 8816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:20.621508 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:20.624519 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:20.625426 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:20.625581 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:20.637514 (+ 11933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:20.637529 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:20.637535 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:20.637599 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:20.643090 (+ 5491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:20.644219 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:20.644226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:20.644230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:20.644437 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:20.645093 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:20.645098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:20.645101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:20.645638 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:20.645770 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:20.647074 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0808 13:12:20.647312 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30790,"thread_start_us":36113,"threads_started":1}
I20260808 13:12:20.650684 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:20.577035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36841 (local address 127.31.59.129:42821)
0808 13:12:20.605299 (+ 28264us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:20.605390 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:20.610805 (+ 5415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:20.612053 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:20.612069 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:20.618752 (+ 6683us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:12:20.620003 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:20.620034 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:20.622013 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:20.622019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:20.624329 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:20.624345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:20.626054 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:20.626065 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:20.626345 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:20.630414 (+ 4069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:12:20.630449 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:12:20.636897 (+ 6448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:12:20.643322 (+ 6425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:20.643332 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:20.643351 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:20.644059 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:20.644610 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:20.644614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:20.644617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:20.644952 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:20.645941 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:12:20.645949 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:12:20.646521 (+ 572us) client_negotiation.cc:769] Sending connection context
0808 13:12:20.648202 (+ 1681us) client_negotiation.cc:240] Negotiation successful
0808 13:12:20.648943 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27831,"thread_start_us":28383,"threads_started":1}
I20260808 13:12:20.652844 11800 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36841
I20260808 13:12:20.653335 11800 heartbeater.cc:464] Registering TS with master...
I20260808 13:12:20.654362 11800 heartbeater.cc:511] Master 127.31.59.190:36841 requested a full tablet report, sending...
I20260808 13:12:20.657330 11571 ts_manager.cc:194] Registered new tserver with Master: ddef6e04ac6a4c24b709b85bec72f2f2 (127.31.59.129:45403)
I20260808 13:12:20.659317 11571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:42821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:12:20.679651 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:42155/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:12:20 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194739, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
W20260808 13:12:21.480298 11807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:12:21.480747 11807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:12:21.480971 11807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:12:21.495277 11807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:12:21.495527 11807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:12:21.495785 11807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:12:21.510860 11807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.59.148:42155/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:12:21.512665 11807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:12:21.514398 11807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:12:21.535059 11823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:21.535272 11822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:21.535326 11825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:12:21.536724 11807 server_base.cc:1034] running on GCE node
I20260808 13:12:21.537487 11807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:12:21.538759 11807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:12:21.540010 11807 hybrid_clock.cc:648] HybridClock initialized: now 1786194741539907 us; error 40 us; skew 500 ppm
I20260808 13:12:21.540638 11807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:12:21 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194741, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:12:21.545773 11807 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:12:21.547510 11807 webserver.cc:460] Webserver started at http://127.31.59.130:42277/ using document root <none> and password file <none>
I20260808 13:12:21.548424 11807 fs_manager.cc:359] Metadata directory not provided
I20260808 13:12:21.548673 11807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:12:21.549142 11807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:12:21 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194741, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:21.569169 11807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9"
format_stamp: "Formatted at 2026-08-08 13:12:21 on dist-test-slave-49h3"
server_key: "3ecf065dfe715e27fc33a2a7854c51ac"
server_key_iv: "9250548358be1cd55c9f05d48a0183a6"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:21.570693 11807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9"
format_stamp: "Formatted at 2026-08-08 13:12:21 on dist-test-slave-49h3"
server_key: "3ecf065dfe715e27fc33a2a7854c51ac"
server_key_iv: "9250548358be1cd55c9f05d48a0183a6"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:21.578697 11807 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260808 13:12:21.600524 11832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:21.602291 11807 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:12:21.602560 11807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9"
format_stamp: "Formatted at 2026-08-08 13:12:21 on dist-test-slave-49h3"
server_key: "3ecf065dfe715e27fc33a2a7854c51ac"
server_key_iv: "9250548358be1cd55c9f05d48a0183a6"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:21.602895 11807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:12:21.620177 11807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:12:21.626394 11807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:12:21.627027 11807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:12:21.629068 11807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:12:21.631122 11807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:12:21.631270 11807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:21.631480 11807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:12:21.631587 11807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:21.663234 11800 heartbeater.cc:503] Master 127.31.59.190:36841 was elected leader, sending a full tablet report...
I20260808 13:12:21.748112 11807 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:43549
I20260808 13:12:21.748204 11952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:43549 every 8 connection(s)
I20260808 13:12:21.750388 11807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:12:21.755991 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 11807
I20260808 13:12:21.756166 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:12:21.756872 31982 external_mini_cluster.cc:1428] Setting key 14e52c77d45b740dd619888daf667b86
Aug 08 13:12:21 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194741, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:12:21.777554 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:21.755019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54493 (local address 127.31.59.190:36841)
0808 13:12:21.756036 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:21.756042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:21.757204 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 13:12:21.759645 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:21.759649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:21.759736 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:21.760014 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:21.762273 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:21.763187 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:21.765803 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:21.766326 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:21.766426 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:21.770980 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:21.770985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:21.770988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:21.771033 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:21.774171 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:21.774913 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:21.774917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:21.774919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:21.775071 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:21.775539 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:21.775543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:21.775545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:21.775842 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:21.776018 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:21.776857 (+ 839us) server_negotiation.cc:299] Negotiation successful
0808 13:12:21.777080 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":2538,"threads_started":1}
I20260808 13:12:21.778595 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:21.755659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36841 (local address 127.31.59.130:54493)
0808 13:12:21.756999 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:21.757072 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:21.759315 (+ 2243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:21.760314 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:21.760324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:21.761027 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:12:21.762070 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:21.762100 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:21.763341 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:21.763347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:21.765650 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:21.765668 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:21.766592 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:21.766600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:21.766923 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:21.767736 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:12:21.767786 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:12:21.770726 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:12:21.774312 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:21.774322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:21.774341 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:21.774787 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:21.775180 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:21.775185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:21.775189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:21.775424 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:21.775983 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:12:21.775991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:12:21.776358 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 13:12:21.776730 (+ 372us) client_negotiation.cc:240] Negotiation successful
0808 13:12:21.777294 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":727,"threads_started":1}
I20260808 13:12:21.780577 11953 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36841
I20260808 13:12:21.780948 11953 heartbeater.cc:464] Registering TS with master...
I20260808 13:12:21.781888 11953 heartbeater.cc:511] Master 127.31.59.190:36841 requested a full tablet report, sending...
I20260808 13:12:21.783457 11571 ts_manager.cc:194] Registered new tserver with Master: 2d1b235dd0fe44aa8e0ec166c946d9e9 (127.31.59.130:43549)
I20260808 13:12:21.784325 11571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:54493
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:12:21.837038 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:42155/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:12:22.448783 11959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:12:22.449260 11959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:12:22.449522 11959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:12:22.459565 11959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:12:22.459815 11959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:12:22.460080 11959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:12:22.473788 11959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.59.148:42155/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:12:22.475164 11959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:12:22.476511 11959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:12:22.494685 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:22.494833 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:12:22.494952 11964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:12:22.496454 11959 server_base.cc:1034] running on GCE node
I20260808 13:12:22.496999 11959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:12:22.498000 11959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:12:22.499203 11959 hybrid_clock.cc:648] HybridClock initialized: now 1786194742499149 us; error 34 us; skew 500 ppm
I20260808 13:12:22.499599 11959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:12:22 dist-test-slave-49h3 krb5kdc[11066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194742, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:12:22.504293 11959 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:12:22.506364 11959 webserver.cc:460] Webserver started at http://127.31.59.131:38595/ using document root <none> and password file <none>
I20260808 13:12:22.507289 11959 fs_manager.cc:359] Metadata directory not provided
I20260808 13:12:22.507522 11959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:12:22.507930 11959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:12:22 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194742, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:12:22.527884 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "96b9e86c3e884f02aaad851d41274dd7"
format_stamp: "Formatted at 2026-08-08 13:12:22 on dist-test-slave-49h3"
server_key: "8be41f6b23ba61811a76d50bc70c5978"
server_key_iv: "d16e0f5e781992f6542f9ea3895cdfd1"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:22.529491 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "96b9e86c3e884f02aaad851d41274dd7"
format_stamp: "Formatted at 2026-08-08 13:12:22 on dist-test-slave-49h3"
server_key: "8be41f6b23ba61811a76d50bc70c5978"
server_key_iv: "d16e0f5e781992f6542f9ea3895cdfd1"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:22.538405 11959 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.002s
I20260808 13:12:22.560353 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:22.562413 11959 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:12:22.562747 11959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "96b9e86c3e884f02aaad851d41274dd7"
format_stamp: "Formatted at 2026-08-08 13:12:22 on dist-test-slave-49h3"
server_key: "8be41f6b23ba61811a76d50bc70c5978"
server_key_iv: "d16e0f5e781992f6542f9ea3895cdfd1"
server_key_version: "kuduclusterkey@0"
I20260808 13:12:22.563109 11959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:12:22.616719 11959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:12:22.622092 11959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:12:22.622712 11959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:12:22.624390 11959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:12:22.626713 11959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:12:22.626896 11959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:22.627110 11959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:12:22.627240 11959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:12:22.706216 11959 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:39913
I20260808 13:12:22.706388 12093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:39913 every 8 connection(s)
I20260808 13:12:22.707932 11959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:12:22.716956 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 11959
I20260808 13:12:22.717111 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:12:22.717650 31982 external_mini_cluster.cc:1428] Setting key a1ce354109904bab305cff21ed267352
Aug 08 13:12:22 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194742, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:12:22.737527 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.711225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:38869 (local address 127.31.59.190:36841)
0808 13:12:22.712041 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:22.712045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:22.713374 (+ 1329us) server_negotiation.cc:407] Connection header received
0808 13:12:22.716217 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:22.716222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:22.716384 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:22.716483 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:22.718651 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.719920 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.722841 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.723812 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.723914 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.729087 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:22.729093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:22.729096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:22.729163 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:22.733740 (+ 4577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:22.734772 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:22.734795 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:22.734799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:22.734960 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:22.735499 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:22.735504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:22.735508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:22.735893 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:22.736035 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:22.736855 (+ 820us) server_negotiation.cc:299] Negotiation successful
0808 13:12:22.737169 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":498,"threads_started":1}
I20260808 13:12:22.738639 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.712003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36841 (local address 127.31.59.131:38869)
0808 13:12:22.713099 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:22.713172 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:22.715899 (+ 2727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:22.716704 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:22.716712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:22.717477 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:12:22.718466 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.718494 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.720063 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.720068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.722623 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.722646 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.724217 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.724228 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.724621 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.725617 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:12:22.725650 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:12:22.728747 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:12:22.733923 (+ 5176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:22.733933 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:22.733949 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:22.734657 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:22.735099 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:12:22.735105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:12:22.735109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:12:22.735372 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:12:22.736031 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:12:22.736039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:12:22.736410 (+ 371us) client_negotiation.cc:769] Sending connection context
0808 13:12:22.736695 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 13:12:22.737227 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":530,"threads_started":1}
I20260808 13:12:22.740625 12094 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36841
I20260808 13:12:22.740942 12094 heartbeater.cc:464] Registering TS with master...
I20260808 13:12:22.741758 12094 heartbeater.cc:511] Master 127.31.59.190:36841 requested a full tablet report, sending...
I20260808 13:12:22.746528 11571 ts_manager.cc:194] Registered new tserver with Master: 96b9e86c3e884f02aaad851d41274dd7 (127.31.59.131:39913)
I20260808 13:12:22.747375 11571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:38869
I20260808 13:12:22.758756 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:12:22.781363 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.764537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.31.59.190:36841)
0808 13:12:22.764754 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:22.764761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:22.765260 (+ 499us) server_negotiation.cc:407] Connection header received
0808 13:12:22.765514 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:22.765520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:22.765646 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:22.765732 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:22.767995 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.769055 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.770912 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.771608 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.771777 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.774942 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:22.774946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:22.774950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:22.774985 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:22.778477 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:22.779181 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:22.779185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:22.779187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:22.779350 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:22.779739 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:22.779744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:22.779747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:22.780114 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:22.780484 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:22.780718 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 13:12:22.780980 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:12:22.786916 11953 heartbeater.cc:503] Master 127.31.59.190:36841 was elected leader, sending a full tablet report...
I20260808 13:12:22.785646 11571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:12:22.796351 11571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:12:22.829571 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.810529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.31.59.130:43549)
0808 13:12:22.817002 (+ 6473us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:22.817013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:22.817050 (+ 37us) server_negotiation.cc:407] Connection header received
0808 13:12:22.818030 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:22.818051 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:22.818374 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:22.818796 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:12:22.820110 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.822309 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.825931 (+ 3622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.827784 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.827961 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.828380 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:22.828932 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 13:12:22.829129 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6210,"thread_start_us":6046,"threads_started":1}
I20260808 13:12:22.830086 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.810250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43549 (local address 127.0.0.1:41564)
0808 13:12:22.811742 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:22.811787 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:22.817775 (+ 5988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:22.818853 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:22.818862 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:22.818899 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:12:22.819846 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.819862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.822492 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.822500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.825777 (+ 3277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.825797 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.828562 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.828568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.828719 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.828755 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:12:22.828868 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 13:12:22.829080 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"mutex_wait_us":1,"thread_start_us":10067,"threads_started":1}
I20260808 13:12:22.831110 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.810500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39913 (local address 127.0.0.1:51224)
0808 13:12:22.819043 (+ 8543us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:22.819066 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:22.819375 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:22.821268 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:22.821273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:22.821298 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:12:22.822031 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.822045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.824080 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.824085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.826969 (+ 2884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.826988 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.830083 (+ 3095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.830091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.830300 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.830332 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:12:22.830410 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:12:22.830538 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8335,"mutex_wait_us":1,"thread_start_us":680,"threads_started":1}
I20260808 13:12:22.831266 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.811034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.31.59.131:39913)
0808 13:12:22.820023 (+ 8989us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:22.820032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:22.820073 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:12:22.820165 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:22.820172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:22.820425 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:22.820828 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:12:22.822308 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.823908 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.827137 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.829082 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.829212 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.829541 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:22.830476 (+ 935us) server_negotiation.cc:299] Negotiation successful
0808 13:12:22.830701 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8821,"thread_start_us":8631,"threads_started":1}
I20260808 13:12:22.832068 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.812892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37398 (local address 127.31.59.129:45403)
0808 13:12:22.817662 (+ 4770us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:22.817668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:22.820971 (+ 3303us) server_negotiation.cc:407] Connection header received
0808 13:12:22.821336 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:22.821344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:22.821640 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:22.821932 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:12:22.822875 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.824826 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.827514 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.829530 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.829649 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.830005 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:22.831425 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0808 13:12:22.831624 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4574,"thread_start_us":4371,"threads_started":1}
I20260808 13:12:22.831297 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:22.812079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45403 (local address 127.0.0.1:37398)
0808 13:12:22.820670 (+ 8591us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:22.820686 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:22.821115 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:22.822153 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:22.822157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:22.822181 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:12:22.822658 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.822671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.824991 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.824996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.827086 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:22.827102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:22.830247 (+ 3145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:22.830256 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:22.830436 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:22.830469 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:12:22.830557 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:12:22.830682 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8412,"mutex_wait_us":21,"thread_start_us":6443,"threads_started":1}
I20260808 13:12:22.834336 11885 tablet_service.cc:1467] Processing CreateTablet for tablet ddccd3458eea45aa88f3ca59018164d4 (DEFAULT_TABLE table=test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:12:22.836211 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddccd3458eea45aa88f3ca59018164d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:12:22.837699 12026 tablet_service.cc:1467] Processing CreateTablet for tablet ddccd3458eea45aa88f3ca59018164d4 (DEFAULT_TABLE table=test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:12:22.839483 12026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddccd3458eea45aa88f3ca59018164d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:12:22.840638 11731 tablet_service.cc:1467] Processing CreateTablet for tablet ddccd3458eea45aa88f3ca59018164d4 (DEFAULT_TABLE table=test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:12:22.842262 11731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddccd3458eea45aa88f3ca59018164d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:12:22.857874 12113 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Bootstrap starting.
I20260808 13:12:22.859513 12115 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Bootstrap starting.
I20260808 13:12:22.863467 12113 tablet_bootstrap.cc:654] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Neither blocks nor log segments found. Creating new log.
I20260808 13:12:22.865144 12116 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Bootstrap starting.
I20260808 13:12:22.865908 12113 log.cc:824] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Log is configured to *not* fsync() on all Append() calls
I20260808 13:12:22.866246 12115 tablet_bootstrap.cc:654] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Neither blocks nor log segments found. Creating new log.
I20260808 13:12:22.868705 12115 log.cc:824] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:12:22.872223 12116 tablet_bootstrap.cc:654] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Neither blocks nor log segments found. Creating new log.
I20260808 13:12:22.872629 12113 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: No bootstrap required, opened a new log
I20260808 13:12:22.872956 12113 ts_tablet_manager.cc:1397] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260808 13:12:22.873080 12115 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: No bootstrap required, opened a new log
I20260808 13:12:22.873467 12115 ts_tablet_manager.cc:1397] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260808 13:12:22.874099 12116 log.cc:824] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:12:22.878328 12116 tablet_bootstrap.cc:492] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: No bootstrap required, opened a new log
I20260808 13:12:22.878639 12116 ts_tablet_manager.cc:1397] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260808 13:12:22.888108 12113 raft_consensus.cc:348] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.888469 12115 raft_consensus.cc:348] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.888985 12115 raft_consensus.cc:374] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:12:22.889045 12113 raft_consensus.cc:374] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:12:22.889158 12115 raft_consensus.cc:729] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9e86c3e884f02aaad851d41274dd7, State: Initialized, Role: FOLLOWER
I20260808 13:12:22.889314 12113 raft_consensus.cc:729] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1b235dd0fe44aa8e0ec166c946d9e9, State: Initialized, Role: FOLLOWER
I20260808 13:12:22.890293 12115 consensus_queue.cc:260] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.890882 12113 consensus_queue.cc:260] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.894198 12116 raft_consensus.cc:348] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.894878 12116 raft_consensus.cc:374] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:12:22.895066 12116 raft_consensus.cc:729] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddef6e04ac6a4c24b709b85bec72f2f2, State: Initialized, Role: FOLLOWER
I20260808 13:12:22.895866 12116 consensus_queue.cc:260] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:22.896181 12113 ts_tablet_manager.cc:1428] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Time spent starting tablet: real 0.023s user 0.009s sys 0.014s
I20260808 13:12:22.897607 12094 heartbeater.cc:503] Master 127.31.59.190:36841 was elected leader, sending a full tablet report...
I20260808 13:12:22.898624 12115 ts_tablet_manager.cc:1428] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Time spent starting tablet: real 0.025s user 0.019s sys 0.005s
I20260808 13:12:22.904318 12116 ts_tablet_manager.cc:1428] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Time spent starting tablet: real 0.026s user 0.007s sys 0.017s
W20260808 13:12:22.960819 12095 tablet.cc:2369] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:12:23.004827 11954 tablet.cc:2369] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:12:23.120957 11801 tablet.cc:2369] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:12:23.132014 12122 raft_consensus.cc:482] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:12:23.132537 12122 raft_consensus.cc:504] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:23.135063 12122 leader_election.cc:288] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d1b235dd0fe44aa8e0ec166c946d9e9 (127.31.59.130:43549), 96b9e86c3e884f02aaad851d41274dd7 (127.31.59.131:39913)
I20260808 13:12:23.146762 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.135596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:58405 (local address 127.31.59.131:39913)
0808 13:12:23.135814 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.135821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.138138 (+ 2317us) server_negotiation.cc:407] Connection header received
0808 13:12:23.138335 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.138341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.138419 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.138486 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:12:23.139800 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.140924 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.144163 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.145376 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.145582 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.145656 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.146302 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.146434 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:12:23.146800 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.135910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:42461 (local address 127.31.59.130:43549)
0808 13:12:23.136118 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.136123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.138144 (+ 2021us) server_negotiation.cc:407] Connection header received
0808 13:12:23.138395 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.138400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.138462 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.138550 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:12:23.139714 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.140867 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.143882 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.145364 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.145564 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.145627 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.146306 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.146435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:12:23.147336 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.136200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39913 (local address 127.31.59.129:58405)
0808 13:12:23.137843 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:23.137859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:23.138198 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:23.138524 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:23.138529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:23.138554 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:12:23.139628 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:23.139643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.141152 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.141156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:23.144012 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:23.144027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.145934 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.145941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:23.146115 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.146158 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 13:12:23.146234 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 13:12:23.146376 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"mutex_wait_us":31,"thread_start_us":1274,"threads_started":1}
I20260808 13:12:23.147392 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.136231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:43549 (local address 127.31.59.129:42461)
0808 13:12:23.137922 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0808 13:12:23.137936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:12:23.138253 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:12:23.138581 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:12:23.138585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:12:23.138609 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:12:23.139423 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:23.139435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.140990 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.140996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:23.143676 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:12:23.143689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.145870 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.145877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:12:23.146147 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.146171 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:12:23.146258 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 13:12:23.146388 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":31,"thread_start_us":851,"threads_started":1}
I20260808 13:12:23.148983 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddccd3458eea45aa88f3ca59018164d4" candidate_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9e86c3e884f02aaad851d41274dd7" is_pre_election: true
I20260808 13:12:23.149088 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddccd3458eea45aa88f3ca59018164d4" candidate_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" is_pre_election: true
I20260808 13:12:23.149670 12047 raft_consensus.cc:2454] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddef6e04ac6a4c24b709b85bec72f2f2 in term 0.
I20260808 13:12:23.150128 11907 raft_consensus.cc:2454] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddef6e04ac6a4c24b709b85bec72f2f2 in term 0.
I20260808 13:12:23.150514 11683 leader_election.cc:302] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b9e86c3e884f02aaad851d41274dd7, ddef6e04ac6a4c24b709b85bec72f2f2; no voters:
I20260808 13:12:23.151012 12122 raft_consensus.cc:2781] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:12:23.151170 12122 raft_consensus.cc:482] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:12:23.151291 12122 raft_consensus.cc:3037] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:12:23.153774 12122 raft_consensus.cc:504] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:23.154404 12122 leader_election.cc:288] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [CANDIDATE]: Term 1 election: Requested vote from peers 2d1b235dd0fe44aa8e0ec166c946d9e9 (127.31.59.130:43549), 96b9e86c3e884f02aaad851d41274dd7 (127.31.59.131:39913)
I20260808 13:12:23.154757 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddccd3458eea45aa88f3ca59018164d4" candidate_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9"
I20260808 13:12:23.154971 11907 raft_consensus.cc:3037] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:12:23.155061 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddccd3458eea45aa88f3ca59018164d4" candidate_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9e86c3e884f02aaad851d41274dd7"
I20260808 13:12:23.155369 12047 raft_consensus.cc:3037] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:12:23.158417 11907 raft_consensus.cc:2454] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddef6e04ac6a4c24b709b85bec72f2f2 in term 1.
I20260808 13:12:23.158850 11682 leader_election.cc:302] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1b235dd0fe44aa8e0ec166c946d9e9, ddef6e04ac6a4c24b709b85bec72f2f2; no voters:
I20260808 13:12:23.159210 12122 raft_consensus.cc:2781] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:12:23.159531 12047 raft_consensus.cc:2454] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddef6e04ac6a4c24b709b85bec72f2f2 in term 1.
I20260808 13:12:23.160055 12122 raft_consensus.cc:686] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 1 LEADER]: Becoming Leader. State: Replica: ddef6e04ac6a4c24b709b85bec72f2f2, State: Running, Role: LEADER
I20260808 13:12:23.160737 12122 consensus_queue.cc:237] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } }
I20260808 13:12:23.166810 11571 catalog_manager.cc:5495] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddef6e04ac6a4c24b709b85bec72f2f2 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddef6e04ac6a4c24b709b85bec72f2f2" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45403 } health_report { overall_health: HEALTHY } } }
I20260808 13:12:23.207367 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.200863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.31.59.129:45403)
0808 13:12:23.201177 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.201183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.201225 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:12:23.201534 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.201538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.201673 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.201808 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:12:23.202200 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.203438 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.205072 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.205677 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.205780 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.206292 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:12:23.206864 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:12:23.206986 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.207055 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.207122 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:12:23.218500 12047 raft_consensus.cc:1261] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 1 FOLLOWER]: Refusing update from remote peer ddef6e04ac6a4c24b709b85bec72f2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:12:23.218663 11907 raft_consensus.cc:1261] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 1 FOLLOWER]: Refusing update from remote peer ddef6e04ac6a4c24b709b85bec72f2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:12:23.219623 12122 consensus_queue.cc:1035] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1b235dd0fe44aa8e0ec166c946d9e9" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:12:23.220074 12125 consensus_queue.cc:1035] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b9e86c3e884f02aaad851d41274dd7" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:12:23.252914 11571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:12:23.253274 11571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:12:23.257417 11571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6140b16e0706443e823f9bc7a1da45dc: Sending DeleteTablet for 3 replicas of tablet ddccd3458eea45aa88f3ca59018164d4
I20260808 13:12:23.258888 11731 tablet_service.cc:1514] Processing DeleteTablet for tablet ddccd3458eea45aa88f3ca59018164d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:12:23 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:37398
I20260808 13:12:23.258841 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet ddccd3458eea45aa88f3ca59018164d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:12:23 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:41564
I20260808 13:12:23.260522 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 11657
I20260808 13:12:23.262019 12026 tablet_service.cc:1514] Processing DeleteTablet for tablet ddccd3458eea45aa88f3ca59018164d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:12:23 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51224
I20260808 13:12:23.268147 12140 tablet_replica.cc:330] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: stopping tablet replica
I20260808 13:12:23.268749 12140 raft_consensus.cc:2229] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:12:23.269315 12140 raft_consensus.cc:2258] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:12:23.275040 12140 ts_tablet_manager.cc:1905] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:12:23.276415 12141 tablet_replica.cc:330] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: stopping tablet replica
I20260808 13:12:23.276994 12141 raft_consensus.cc:2229] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:12:23.277964 12141 raft_consensus.cc:2258] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:12:23.278911 12138 tablet_replica.cc:330] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: stopping tablet replica
I20260808 13:12:23.279364 12138 raft_consensus.cc:2229] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:12:23.279940 12138 raft_consensus.cc:2258] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:12:23.280594 12141 ts_tablet_manager.cc:1905] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:12:23.283619 12138 ts_tablet_manager.cc:1905] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:12:23.285348 12140 ts_tablet_manager.cc:1918] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:12:23.285609 12140 log.cc:1196] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/ddccd3458eea45aa88f3ca59018164d4
I20260808 13:12:23.286242 12140 ts_tablet_manager.cc:1939] T ddccd3458eea45aa88f3ca59018164d4 P 96b9e86c3e884f02aaad851d41274dd7: Deleting consensus metadata
I20260808 13:12:23.288661 11558 catalog_manager.cc:4841] TS 96b9e86c3e884f02aaad851d41274dd7 (127.31.59.131:39913): tablet ddccd3458eea45aa88f3ca59018164d4 (table test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]) successfully deleted
I20260808 13:12:23.291282 12138 ts_tablet_manager.cc:1918] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:12:23.291266 12141 ts_tablet_manager.cc:1918] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:12:23.291479 12141 log.cc:1196] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/ddccd3458eea45aa88f3ca59018164d4
I20260808 13:12:23.291512 12138 log.cc:1196] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/ddccd3458eea45aa88f3ca59018164d4
I20260808 13:12:23.292197 12141 ts_tablet_manager.cc:1939] T ddccd3458eea45aa88f3ca59018164d4 P ddef6e04ac6a4c24b709b85bec72f2f2: Deleting consensus metadata
I20260808 13:12:23.292222 12138 ts_tablet_manager.cc:1939] T ddccd3458eea45aa88f3ca59018164d4 P 2d1b235dd0fe44aa8e0ec166c946d9e9: Deleting consensus metadata
I20260808 13:12:23.294186 11558 catalog_manager.cc:4841] TS ddef6e04ac6a4c24b709b85bec72f2f2 (127.31.59.129:45403): tablet ddccd3458eea45aa88f3ca59018164d4 (table test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]) successfully deleted
I20260808 13:12:23.294198 11557 catalog_manager.cc:4841] TS 2d1b235dd0fe44aa8e0ec166c946d9e9 (127.31.59.130:43549): tablet ddccd3458eea45aa88f3ca59018164d4 (table test-table [id=89f4b6f974b344818fd0780d6b4bfe5b]) successfully deleted
Aug 08 13:12:23 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:12:23.304527 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.262259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.31.59.129:45403)
0808 13:12:23.262456 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.262462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.274299 (+ 11837us) server_negotiation.cc:407] Connection header received
0808 13:12:23.274532 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.274539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.274661 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.274777 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:23.276663 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.277949 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.285790 (+ 7841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.286689 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.286803 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.297171 (+ 10368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:23.297183 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:23.297199 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:23.297262 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:23.301045 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.302096 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.302103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.302106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.302310 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.302833 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.302836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.302838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.303264 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:23.303627 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.303995 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.304187 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:12:23.305696 11793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:12:23.418776 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 11657
I20260808 13:12:23.438083 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 11807
Aug 08 13:12:23 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:12:23.452503 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.438789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.31.59.130:43549)
0808 13:12:23.438941 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.438946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.438984 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:12:23.439135 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.439139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.439202 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.439285 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:23.440592 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.441310 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.442654 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.443063 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.443153 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.447146 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:23.447162 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:23.447178 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:23.447217 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:23.450023 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.450655 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.450663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.450665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.450828 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.451122 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.451125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.451127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.451470 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:23.451717 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.451982 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.452171 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 13:12:23.453105 11948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:12:23.541805 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 11807
I20260808 13:12:23.559959 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 11959
Aug 08 13:12:23 dist-test-slave-49h3 krb5kdc[11066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:12:23.574293 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:12:23.560677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.31.59.131:39913)
0808 13:12:23.560874 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:12:23.560879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:12:23.560897 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:12:23.561043 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:12:23.561047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:12:23.561122 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:12:23.561201 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:12:23.562491 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.563184 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.564406 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:12:23.564772 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:12:23.564858 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:12:23.568890 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:12:23.568905 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:12:23.568921 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:12:23.568956 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:12:23.571649 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.572265 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.572273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.572275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.572410 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:12:23.572747 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:12:23.572752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:12:23.572754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:12:23.573092 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:12:23.573370 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 13:12:23.573749 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 13:12:23.573960 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:12:23.574925 12089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:12:23.662423 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 11959
I20260808 13:12:23.680145 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 11536
I20260808 13:12:23.681344 11603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:12:23.799031 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 11536
I20260808 13:12:23.821839 31982 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 13:12:24.155848 31982 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 13:12:24.156080 31982 mini_ranger.cc:66] Stopping Ranger...
I20260808 13:12:24.502210 31982 mini_ranger.cc:68] Stopped Ranger
2026-08-08 13:12:24.502 UTC [11096] LOG: received smart shutdown request
2026-08-08 13:12:24.506 UTC [11096] LOG: background worker "logical replication launcher" (PID 11105) exited with exit code 1
2026-08-08 13:12:24.506 UTC [11100] LOG: shutting down
2026-08-08 13:12:24.645 UTC [11096] LOG: database system is shut down
2026-08-08T13:12:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51438 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:12:24 dist-test-slave-49h3 krb5kdc[12154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:12:24 dist-test-slave-49h3 krb5kdc[12154](info): set up 2 sockets
Aug 08 13:12:24 dist-test-slave-49h3 krb5kdc[12154](info): commencing operation
krb5kdc: starting...
W20260808 13:12:25.842013 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:12:26 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:12:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangeradmin_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangeradmin_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangerlookup_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangerlookup_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
I20260808 13:12:26.222167 31982 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskePcvnh/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/postgres -l logfile start
2026-08-08 13:12:28.468 UTC [12187] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 13:12:28.468 UTC [12187] LOG: listening on IPv4 address "127.31.59.148", port 60223
2026-08-08 13:12:28.471 UTC [12187] LOG: listening on Unix socket "/tmp/.s.PGSQL.60223"
2026-08-08 13:12:28.493 UTC [12189] LOG: database system was shut down at 2026-08-08 13:12:27 UTC
2026-08-08 13:12:28.496 UTC [12187] LOG: database system is ready to accept connections
I20260808 13:12:29.445194 31982 mini_postgres.cc:96] Postgres bound to 60223
2026-08-08 13:12:29.468 UTC [12198] FATAL: database "slave" does not exist
127.31.59.148:60223 - accepting connections
I20260808 13:12:29.468694 31982 mini_ranger.cc:161] Starting Ranger...
I20260808 13:12:29.499001 31982 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 13:12:29.917028 31982 mini_ranger.cc:87] Created ranger Postgres database
I20260808 13:12:29.917160 31982 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin
2026-08-08 13:12:30,213 [I] DB FLAVOR :POSTGRES
2026-08-08 13:12:30,214 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:12:30,214 [I] Checking connection..
2026-08-08 13:12:30,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 13:12:30,479 [I] Checking connection passed.
2026-08-08 13:12:30,479 [I] --------- Verifying version history table ---------
2026-08-08 13:12:30,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:12:30,743 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:12:30,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:12:31,012 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:12:31,012 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 13:12:31,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 13:12:31.271 UTC [12267] WARNING: there is no transaction in progress
2026-08-08 13:12:31,277 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 13:12:31,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:12:31,534 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 13:12:31,534 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:12:31,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 13:12:31,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 13:12:32,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-08 13:12:32,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:12:32,565 [I] Table x_portal_user does not exist in database ranger
2026-08-08 13:12:32,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:12:32,811 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 13:12:32,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:12:33,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:12:33,300 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 13:12:33,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 13:12:34.184 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.199 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.213 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.226 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.243 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.259 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.308 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.313 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.326 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.335 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.344 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.357 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.366 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.372 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.377 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34.383 UTC [12385] WARNING: there is no transaction in progress
2026-08-08 13:12:34,933 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 13:12:34,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 13:12:35,183 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 13:12:35,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:12:35,443 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 13:12:35,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:12:35,676 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 13:12:35,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:12:35,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:12:36,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-08 13:12:36,448 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 13:12:36,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 13:12:36,678 [I] DB_PATCHES have already been applied
I20260808 13:12:36.684656 31982 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskePcvnh/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 13:12:36.684751 31982 mini_ranger.cc:193] Using host: 127.31.59.148
I20260808 13:12:36.705364 31982 mini_ranger.cc:227] Ranger admin URL: http://127.31.59.148:34271
Aug 08, 2026 1:12:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 1:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 13:12:37.785770 31982 mini_ranger.cc:160] Time spent starting Ranger: real 8.317s user 0.006s sys 0.055s
Aug 08, 2026 1:12:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 1:12:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 1:12:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 1:12:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 1:12:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 13:12:58.183369 31982 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.59.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.59.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangerkms_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.59.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/rangerkms_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.59.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/HTTP_127.31.59.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 13:12:58 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194778, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 13:12:58.386562 31982 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 13:12:58.412204 31982 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 13:12:59.216022 31982 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 13:12:59.216143 31982 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms
I20260808 13:12:59.216180 31982 mini_ranger_kms.cc:227] Using postgres at 127.31.59.148:60223
2026-08-08 13:12:59,584 [I] DB FLAVOR :POSTGRES
2026-08-08 13:12:59,585 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:12:59,585 [I] Checking connection
2026-08-08 13:12:59,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 13:12:59,839 [I] connection success
2026-08-08 13:12:59,839 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 13:12:59,839 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 13:12:59,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 13:13:00,090 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 13:13:00,090 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:13:00,090 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 13:13:00,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.148:60223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 13:13:00,347 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 13:13:00.822233 31982 mini_ranger_kms.cc:310] Created kms service
I20260808 13:13:01.036093 31982 mini_ranger_kms.cc:326] Created kudu user
I20260808 13:13:01.081220 31982 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 13:13:01 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194778, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:01.503791 31982 mini_ranger_kms.cc:384] Added ranger policy
I20260808 13:13:01.504117 31982 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms:/tmp/dist-test-taskePcvnh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskePcvnh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 13:13:01.504168 31982 mini_ranger_kms.cc:240] Using host: 127.31.59.148
I20260808 13:13:01.526499 31982 mini_ranger_kms.cc:276] Ranger KMS PID: 12577
I20260808 13:13:01.526633 31982 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.59.148:49399
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 1:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 13:13:03.661092 31982 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.275s user 0.085s sys 0.324s
I20260808 13:13:03.661332 31982 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 13:13:03.661517 31982 mini_ranger_kms.cc:401] 127.31.59.148:49399/kms/v1/keys
Aug 08 13:13:03 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194783, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 13:13:04 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194783, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
Aug 08 13:13:04 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194783, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.148@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
Aug 08 13:13:06 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:13:06.882930 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44719
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:44719
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:07.221557 12632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:07.221843 12632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:07.221974 12632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:07.229267 12632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:07.229382 12632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:07.229470 12632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:07.229552 12632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:07.229605 12632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:07.239799 12632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.59.148:49399/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:44719
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:07.241154 12632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:07.242547 12632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:07.254290 12638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:07.254321 12641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:07.254354 12639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:07.255178 12632 server_base.cc:1034] running on GCE node
I20260808 13:13:07.255867 12632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:07.257176 12632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:07.258368 12632 hybrid_clock.cc:648] HybridClock initialized: now 1786194787258327 us; error 33 us; skew 500 ppm
I20260808 13:13:07.258669 12632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:07 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194787, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:07.264756 12632 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:07.266291 12632 webserver.cc:460] Webserver started at http://127.31.59.190:42001/ using document root <none> and password file <none>
I20260808 13:13:07.266979 12632 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:07.267088 12632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:07.267346 12632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:13:07 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194787, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:07.318199 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "8af5e4bf0eae4b0799d07b69b26829f7"
format_stamp: "Formatted at 2026-08-08 13:13:07 on dist-test-slave-49h3"
server_key: "4650653587e86cd677fcfbf3f7fe8806"
server_key_iv: "62a3e98d4041bfcd7d3e423f8693e93d"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:07.319188 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "8af5e4bf0eae4b0799d07b69b26829f7"
format_stamp: "Formatted at 2026-08-08 13:13:07 on dist-test-slave-49h3"
server_key: "4650653587e86cd677fcfbf3f7fe8806"
server_key_iv: "62a3e98d4041bfcd7d3e423f8693e93d"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:07.324674 12632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:13:07.341670 12649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:07.343253 12632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:07.343446 12632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "8af5e4bf0eae4b0799d07b69b26829f7"
format_stamp: "Formatted at 2026-08-08 13:13:07 on dist-test-slave-49h3"
server_key: "4650653587e86cd677fcfbf3f7fe8806"
server_key_iv: "62a3e98d4041bfcd7d3e423f8693e93d"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:07.343677 12632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:07.374480 12632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:07.378633 12632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:07.378952 12632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:07.398202 12632 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:44719
I20260808 13:13:07.398232 12704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:44719 every 8 connection(s)
I20260808 13:13:07.399601 12632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:07.401824 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 12632
I20260808 13:13:07.402058 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:13:07.402932 31982 external_mini_cluster.cc:1428] Setting key 6c7a4f1fadc246fc5dd6d1d9ddd4a22c
I20260808 13:13:07.404072 12705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:13:07 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:07.416154 12705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Bootstrap starting.
I20260808 13:13:07.419517 12705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:07.420545 12705 log.cc:824] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:07.423528 12705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: No bootstrap required, opened a new log
I20260808 13:13:07.424216 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:07.405010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.31.59.190:44719)
0808 13:13:07.405844 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:07.405857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:07.405896 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:13:07.407465 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:07.407487 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:07.407878 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:07.408244 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:07.409640 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:07.411001 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:07.412282 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:07.412969 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:07.413165 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:07.417210 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:07.417218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:07.417233 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:07.417270 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:07.420232 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:07.420896 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:07.420903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:07.420909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:07.421103 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:07.421495 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:07.421499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:07.421501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:07.422039 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:07.422347 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:07.422774 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 13:13:07.423158 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":418,"threads_started":1}
I20260808 13:13:07.431905 12705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:07.432263 12705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:07.432354 12705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af5e4bf0eae4b0799d07b69b26829f7, State: Initialized, Role: FOLLOWER
I20260808 13:13:07.432914 12705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:07.433142 12705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:07.433276 12705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:07.433490 12705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:07.435575 12705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:07.436035 12705 leader_election.cc:302] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8af5e4bf0eae4b0799d07b69b26829f7; no voters:
I20260808 13:13:07.436576 12705 leader_election.cc:288] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:13:07.436690 12710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:07.437350 12710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8af5e4bf0eae4b0799d07b69b26829f7, State: Running, Role: LEADER
I20260808 13:13:07.437922 12710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:07.438571 12705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:07.440719 12712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af5e4bf0eae4b0799d07b69b26829f7. Latest consensus state: current_term: 1 leader_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } } }
I20260808 13:13:07.440716 12711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } } }
I20260808 13:13:07.441296 12711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:07.441296 12712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:07.442184 12717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:07.447644 12717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:07.456334 12705 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 13:13:07.458758 12717 catalog_manager.cc:1347] Generated new cluster ID: d8cc2e24a82c4d94aea8b2e090cf3a93
I20260808 13:13:07.458879 12717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:07.485394 12717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:13:07.487625 12717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:07.496155 12717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Generated new TSK 0
I20260808 13:13:07.497275 12717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:13:08.309587 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:08 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 13:13:08.771428 12750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:08.771924 12750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:08.772114 12750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:08.783610 12750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:08.783787 12750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:08.783987 12750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:08.799690 12750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:08.801608 12750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:08.803505 12750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:08.820936 12761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:08.820940 12762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:08.828157 12764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:08.829169 12750 server_base.cc:1034] running on GCE node
I20260808 13:13:08.829982 12750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:08.831290 12750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:08.832581 12750 hybrid_clock.cc:648] HybridClock initialized: now 1786194788832504 us; error 63 us; skew 500 ppm
I20260808 13:13:08.832970 12750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:08 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:08.837136 12750 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:08.839913 12750 webserver.cc:460] Webserver started at http://127.31.59.129:41509/ using document root <none> and password file <none>
I20260808 13:13:08.840864 12750 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:08.841039 12750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:08.841464 12750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:13:08 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:08.862198 12750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "bf207803a986452d934e68a6acb95124"
format_stamp: "Formatted at 2026-08-08 13:13:08 on dist-test-slave-49h3"
server_key: "f19a5187269cb2eb8a79598898a7536b"
server_key_iv: "50f7f85b4590767c000ceca413835d34"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:08.863303 12750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "bf207803a986452d934e68a6acb95124"
format_stamp: "Formatted at 2026-08-08 13:13:08 on dist-test-slave-49h3"
server_key: "f19a5187269cb2eb8a79598898a7536b"
server_key_iv: "50f7f85b4590767c000ceca413835d34"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:08.880925 12750 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.008s sys 0.008s
I20260808 13:13:08.902146 12773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:08.904138 12750 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.003s
I20260808 13:13:08.904364 12750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "bf207803a986452d934e68a6acb95124"
format_stamp: "Formatted at 2026-08-08 13:13:08 on dist-test-slave-49h3"
server_key: "f19a5187269cb2eb8a79598898a7536b"
server_key_iv: "50f7f85b4590767c000ceca413835d34"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:08.904758 12750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:08.963856 12750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:08.968259 12750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:08.968595 12750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:08.969724 12750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:08.971297 12750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:08.971375 12750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:08.971478 12750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:08.971534 12750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:09.054186 12750 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45443
I20260808 13:13:09.054209 12891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45443 every 8 connection(s)
I20260808 13:13:09.056128 12750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:09.063723 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 12750
I20260808 13:13:09.063961 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:13:09.064818 31982 external_mini_cluster.cc:1428] Setting key dbb07bad0cb698c1a05373a2b28d7941
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
I20260808 13:13:09.103442 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:09.059934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:46401 (local address 127.31.59.190:44719)
0808 13:13:09.081322 (+ 21388us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:09.081328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:09.081356 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:13:09.082038 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:09.082057 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:09.082168 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:09.082272 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:09.085641 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.086838 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.089513 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.090161 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.090278 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:09.095267 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:09.095275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:09.095279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:09.095339 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:09.099292 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:09.100348 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:09.100354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:09.100357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:09.100530 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:09.101125 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:09.101130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:09.101134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:09.101546 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:09.101687 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:09.102775 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0808 13:13:09.103016 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21242,"thread_start_us":21401,"threads_started":1}
I20260808 13:13:09.105952 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:09.060665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.129:46401)
0808 13:13:09.078935 (+ 18270us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:09.079003 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:09.081582 (+ 2579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:09.082739 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:09.082753 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:09.083434 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:09.084266 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:09.084292 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.087106 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.087111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:09.089266 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:09.089284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.090694 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.090703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:09.091019 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:09.091911 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:09.091951 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:09.094928 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:09.099523 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:09.099533 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:09.099552 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:09.100190 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:09.100698 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:09.100703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:09.100706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:09.100965 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:09.101788 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:09.101797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:09.102297 (+ 500us) client_negotiation.cc:769] Sending connection context
0808 13:13:09.103774 (+ 1477us) client_negotiation.cc:240] Negotiation successful
0808 13:13:09.104360 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17932,"thread_start_us":17673,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
I20260808 13:13:09.108139 12893 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:09.108501 12893 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:09.109398 12893 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:09.112186 12667 ts_manager.cc:194] Registered new tserver with Master: bf207803a986452d934e68a6acb95124 (127.31.59.129:45443)
I20260808 13:13:09.114368 12667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:46401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:13:09.151492 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
W20260808 13:13:09.614287 12899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:09.615223 12899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:09.615443 12899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:09.626745 12899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:09.626869 12899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:09.627048 12899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:09.640625 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:09.642386 12899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:09.644090 12899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:09.661684 12906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:09.662472 12907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:09.676637 12899 server_base.cc:1034] running on GCE node
W20260808 13:13:09.677666 12910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:09.678659 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:09.679996 12899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:09.681265 12899 hybrid_clock.cc:648] HybridClock initialized: now 1786194789681202 us; error 59 us; skew 500 ppm
I20260808 13:13:09.681619 12899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:09.686699 12899 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:09.713263 12899 webserver.cc:460] Webserver started at http://127.31.59.130:37079/ using document root <none> and password file <none>
I20260808 13:13:09.714334 12899 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:09.714558 12899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:09.715029 12899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:09.734520 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "8f3a4115850549bd86ab419786124287"
format_stamp: "Formatted at 2026-08-08 13:13:09 on dist-test-slave-49h3"
server_key: "e2ddb234e14ae58d2fdb229ffd18e41a"
server_key_iv: "b3c16aae6ad0c939e921a99b47f9ab3e"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:09.735415 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "8f3a4115850549bd86ab419786124287"
format_stamp: "Formatted at 2026-08-08 13:13:09 on dist-test-slave-49h3"
server_key: "e2ddb234e14ae58d2fdb229ffd18e41a"
server_key_iv: "b3c16aae6ad0c939e921a99b47f9ab3e"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:09.741233 12899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:13:09.774242 12917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:09.776492 12899 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.005s sys 0.010s
I20260808 13:13:09.776837 12899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "8f3a4115850549bd86ab419786124287"
format_stamp: "Formatted at 2026-08-08 13:13:09 on dist-test-slave-49h3"
server_key: "e2ddb234e14ae58d2fdb229ffd18e41a"
server_key_iv: "b3c16aae6ad0c939e921a99b47f9ab3e"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:09.777237 12899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:09.814247 12899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:09.818509 12899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:09.818854 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:09.819998 12899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:09.821496 12899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:09.821583 12899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:09.821707 12899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:09.821768 12899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:09.867950 12899 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44843
I20260808 13:13:09.867967 13043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44843 every 8 connection(s)
I20260808 13:13:09.869522 12899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:09.869944 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 12899
I20260808 13:13:09.870114 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:13:09.870731 31982 external_mini_cluster.cc:1428] Setting key c8f7981ecb60cfa705f108b5d732ce30
Aug 08 13:13:09 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
I20260808 13:13:09.921226 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:09.893754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:42681 (local address 127.31.59.190:44719)
0808 13:13:09.897932 (+ 4178us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:09.897940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:09.901507 (+ 3567us) server_negotiation.cc:407] Connection header received
0808 13:13:09.902903 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:09.902909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:09.903003 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:09.903079 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:09.905405 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.906498 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.908787 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.909354 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.909477 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:09.913977 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:09.913983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:09.913988 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:09.914028 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:09.917346 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:09.918310 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:09.918316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:09.918320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:09.918470 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:09.919037 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:09.919042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:09.919045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:09.919419 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:09.919550 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:09.920603 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0808 13:13:09.920843 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3995,"thread_start_us":3853,"threads_started":1}
I20260808 13:13:09.922317 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:09.893062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.130:42681)
0808 13:13:09.900102 (+ 7040us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:09.900151 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:09.902563 (+ 2412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:09.903678 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:09.903690 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:09.904332 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:09.905198 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:09.905225 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.906712 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.906717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:09.908578 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:09.908601 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:09.909831 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:09.909838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:09.910100 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:09.910899 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:09.910928 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:09.913722 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:09.917551 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:09.917560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:09.917578 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:09.918168 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:09.918624 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:09.918629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:09.918632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:09.918895 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:09.919654 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:09.919700 (+ 46us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:09.920211 (+ 511us) client_negotiation.cc:769] Sending connection context
0808 13:13:09.920461 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 13:13:09.920991 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6742,"thread_start_us":6485,"threads_started":1}
I20260808 13:13:09.923662 13044 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:09.923966 13044 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:09.924645 13044 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:09.926093 12667 ts_manager.cc:194] Registered new tserver with Master: 8f3a4115850549bd86ab419786124287 (127.31.59.130:44843)
I20260808 13:13:09.926926 12667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:42681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:13:09.969238 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:10.118283 12893 heartbeater.cc:503] Master 127.31.59.190:44719 was elected leader, sending a full tablet report...
W20260808 13:13:10.304944 13050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:10.305249 13050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:10.305397 13050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:10.312764 13050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:10.312888 13050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:10.313040 13050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:10.322795 13050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:10.324098 13050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:10.325388 13050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:10.338025 13055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:10.338192 13058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:10.338579 13056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:10.338922 13050 server_base.cc:1034] running on GCE node
I20260808 13:13:10.339510 13050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:10.340418 13050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:10.341609 13050 hybrid_clock.cc:648] HybridClock initialized: now 1786194790341558 us; error 42 us; skew 500 ppm
I20260808 13:13:10.341920 13050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194790, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:10.345008 13050 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:10.346436 13050 webserver.cc:460] Webserver started at http://127.31.59.131:39631/ using document root <none> and password file <none>
I20260808 13:13:10.347148 13050 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:10.347262 13050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:10.347566 13050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194790, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:10.364187 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "a9e2980d3fab400a9ea9287f4b619010"
format_stamp: "Formatted at 2026-08-08 13:13:10 on dist-test-slave-49h3"
server_key: "8f5859773e097af9bac33c8c37f35d26"
server_key_iv: "f2e530d3d5ad706309c16a6e63204632"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:10.365034 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "a9e2980d3fab400a9ea9287f4b619010"
format_stamp: "Formatted at 2026-08-08 13:13:10 on dist-test-slave-49h3"
server_key: "8f5859773e097af9bac33c8c37f35d26"
server_key_iv: "f2e530d3d5ad706309c16a6e63204632"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:10.370564 13050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:13:10.388118 13066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:10.389710 13050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:13:10.389884 13050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "a9e2980d3fab400a9ea9287f4b619010"
format_stamp: "Formatted at 2026-08-08 13:13:10 on dist-test-slave-49h3"
server_key: "8f5859773e097af9bac33c8c37f35d26"
server_key_iv: "f2e530d3d5ad706309c16a6e63204632"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:10.390105 13050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:10.411677 13050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:10.415319 13050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:10.415601 13050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:10.416550 13050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:10.417881 13050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:10.417958 13050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:10.418079 13050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:10.418133 13050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:10.447780 13050 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:46033
I20260808 13:13:10.447820 13184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:46033 every 8 connection(s)
I20260808 13:13:10.449124 13050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:10.456617 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13050
I20260808 13:13:10.456815 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:13:10.457392 31982 external_mini_cluster.cc:1428] Setting key a572735d142350d390e916a61dd9770c
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194790, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:10.467744 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:10.451561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:45749 (local address 127.31.59.190:44719)
0808 13:13:10.452041 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:10.452046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:10.452845 (+ 799us) server_negotiation.cc:407] Connection header received
0808 13:13:10.454623 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:10.454629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:10.454715 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:10.454811 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:10.456161 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.457214 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.458964 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.459589 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.459698 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:10.462871 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:10.462876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:10.462879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:10.462914 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:10.465225 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.465913 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.465917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.465919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.466059 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.466422 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.466425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.466427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.466667 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:10.466782 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:10.467326 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 13:13:10.467492 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":224,"threads_started":1}
I20260808 13:13:10.468657 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:10.452007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.131:45749)
0808 13:13:10.452690 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:10.452726 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:10.454361 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:10.455048 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:10.455056 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:10.455453 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:10.455998 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:10.456014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.457379 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.457383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:10.458833 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:10.458845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.459810 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.459818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:10.460051 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:10.460673 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:10.460695 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:10.462678 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:10.465369 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:10.465375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:10.465387 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:10.465821 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:10.466136 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:10.466138 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:10.466141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:10.466342 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:10.466754 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:10.466759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:10.467045 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 13:13:10.467258 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 13:13:10.467611 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":271,"threads_started":1}
I20260808 13:13:10.469970 13185 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:10.470263 13185 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:10.471047 13185 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:10.472715 12666 ts_manager.cc:194] Registered new tserver with Master: a9e2980d3fab400a9ea9287f4b619010 (127.31.59.131:46033)
I20260808 13:13:10.473675 12666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:45749
I20260808 13:13:10.484997 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:13:10.486759 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 12750
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:13:10.502018 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:10.487575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.31.59.129:45443)
0808 13:13:10.488145 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:10.488151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:10.488176 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:13:10.488241 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:10.488247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:10.488450 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:10.488575 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:10.489971 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.490972 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.492462 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.492997 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.493087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:10.497136 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:10.497151 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:10.497165 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:10.497203 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:10.499779 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.500366 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.500372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.500375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.500506 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.500782 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.500785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.500787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.501090 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:10.501328 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:10.501544 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 13:13:10.501744 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":257,"threads_started":1}
I20260808 13:13:10.503314 12887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:10.586941 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 12750
I20260808 13:13:10.603910 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 12899
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:13:10.619735 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:10.604783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.31.59.130:44843)
0808 13:13:10.605529 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:10.605536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:10.605562 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:13:10.605626 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:10.605633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:10.605865 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:10.605994 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:10.607264 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.608562 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.609882 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.610515 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.610618 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:10.614555 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:10.614566 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:10.614580 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:10.614614 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:10.617251 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.618054 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.618060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.618062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.618200 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.618498 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.618501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.618503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.618824 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:10.619038 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:10.619241 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 13:13:10.619447 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":438,"threads_started":1}
I20260808 13:13:10.620949 13039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:10.702185 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 12899
I20260808 13:13:10.718529 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13050
Aug 08 13:13:10 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:13:10.734233 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:10.719513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.31.59.131:46033)
0808 13:13:10.720150 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:10.720157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:10.720186 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:13:10.720246 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:10.720253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:10.720500 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:10.720624 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:10.722056 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.723094 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.724243 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:10.724790 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:10.724881 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:10.728997 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:10.729008 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:10.729023 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:10.729058 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:10.731537 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.732179 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.732186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.732188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.732314 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:10.732594 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:10.732597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:10.732598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:10.732938 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:10.733254 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:10.733560 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 13:13:10.733831 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":329,"threads_started":1}
I20260808 13:13:10.735450 13179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:10.816722 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13050
I20260808 13:13:10.830982 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 12632
I20260808 13:13:10.832226 12700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:10.941532 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 12632
I20260808 13:13:10.964075 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44719
--webserver_interface=127.31.59.190
--webserver_port=42001
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:44719
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:11.298379 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:11.298688 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:11.298830 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:11.306100 13199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:11.306228 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:11.306308 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:11.306391 13199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:11.306458 13199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:11.316608 13199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:44719
--ranger_config_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=42001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:11.317984 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:11.319314 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:11.330657 13204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:11.330716 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:11.330750 13205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:11.331632 13199 server_base.cc:1034] running on GCE node
I20260808 13:13:11.332369 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:11.333756 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:11.334951 13199 hybrid_clock.cc:648] HybridClock initialized: now 1786194791334907 us; error 30 us; skew 500 ppm
I20260808 13:13:11.335300 13199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:11 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194791, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:11.338253 13199 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:11.339612 13199 webserver.cc:460] Webserver started at http://127.31.59.190:42001/ using document root <none> and password file <none>
I20260808 13:13:11.340214 13199 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:11.340338 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:13:11 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194791, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:11.416803 13199 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 13:13:11.419765 13217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:11.421064 13199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:13:11.421245 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "8af5e4bf0eae4b0799d07b69b26829f7"
format_stamp: "Formatted at 2026-08-08 13:13:07 on dist-test-slave-49h3"
server_key: "4650653587e86cd677fcfbf3f7fe8806"
server_key_iv: "62a3e98d4041bfcd7d3e423f8693e93d"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:11.422263 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:11.463040 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:11.466764 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:11.467036 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:11.484470 13199 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:44719
I20260808 13:13:11.484489 13272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:44719 every 8 connection(s)
I20260808 13:13:11.485953 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:11.492746 13273 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:13:11.494360 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13199
I20260808 13:13:11.495407 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:45443
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:44719
--webserver_port=41509
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:11.496876 13273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Bootstrap starting.
I20260808 13:13:11.518056 13273 log.cc:824] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:11.527283 13273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:13:11.527802 13273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Bootstrap complete.
I20260808 13:13:11.536885 13273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:11.537873 13273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af5e4bf0eae4b0799d07b69b26829f7, State: Initialized, Role: FOLLOWER
I20260808 13:13:11.538515 13273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:11.538730 13273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:11.538844 13273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:11.539000 13273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:13:11.542871 13273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:11.543358 13273 leader_election.cc:302] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8af5e4bf0eae4b0799d07b69b26829f7; no voters:
I20260808 13:13:11.543983 13273 leader_election.cc:288] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:13:11.544126 13277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:13:11.545236 13277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8af5e4bf0eae4b0799d07b69b26829f7, State: Running, Role: LEADER
I20260808 13:13:11.545897 13277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } }
I20260808 13:13:11.546221 13273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:11.548480 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af5e4bf0eae4b0799d07b69b26829f7. Latest consensus state: current_term: 2 leader_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } } }
I20260808 13:13:11.548882 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:11.549396 13278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af5e4bf0eae4b0799d07b69b26829f7" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44719 } } }
I20260808 13:13:11.549700 13278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:11.556341 13284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:11.562216 13284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:11.563692 13284 catalog_manager.cc:1259] Loaded cluster ID: d8cc2e24a82c4d94aea8b2e090cf3a93
I20260808 13:13:11.563881 13284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:11.571283 13284 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:11.573027 13284 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Loaded TSK: 0
I20260808 13:13:11.574771 13284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:13:11.578254 13273 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 13:13:11.956766 13275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:11.957309 13275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:11.957551 13275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:11.969426 13275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:11.969632 13275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:11.969838 13275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:11.986311 13275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:45443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=41509
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:11.988307 13275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:11.990437 13275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:12.010296 13312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:12.011588 13313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:12.016907 13275 server_base.cc:1034] running on GCE node
W20260808 13:13:12.017388 13315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:12.018065 13275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:12.019127 13275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:12.020357 13275 hybrid_clock.cc:648] HybridClock initialized: now 1786194792020300 us; error 48 us; skew 500 ppm
I20260808 13:13:12.020700 13275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:12.024196 13275 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:12.025954 13275 webserver.cc:460] Webserver started at http://127.31.59.129:41509/ using document root <none> and password file <none>
I20260808 13:13:12.026671 13275 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:12.026806 13275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:12.068346 13275 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.006s sys 0.007s
I20260808 13:13:12.074258 13326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.075896 13275 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260808 13:13:12.076061 13275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "bf207803a986452d934e68a6acb95124"
format_stamp: "Formatted at 2026-08-08 13:13:08 on dist-test-slave-49h3"
server_key: "f19a5187269cb2eb8a79598898a7536b"
server_key_iv: "50f7f85b4590767c000ceca413835d34"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:12.077409 13275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:12.117066 13275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:12.122622 13275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:12.123054 13275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:12.129168 13275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:12.130934 13275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:12.131040 13275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.131166 13275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:12.131246 13275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.180882 13275 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45443
I20260808 13:13:12.180961 13446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45443 every 8 connection(s)
I20260808 13:13:12.182754 13275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:12.189966 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13275
I20260808 13:13:12.190874 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:44843
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:44719
--webserver_port=37079
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:12.235431 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:12.186205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.129:55903)
0808 13:13:12.208427 (+ 22222us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:12.208479 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:12.210871 (+ 2392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:12.214207 (+ 3336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:12.214215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:12.214681 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:12.215299 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:12.215326 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.217371 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.217377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:12.218677 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:12.218690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.220238 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.220247 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:12.220530 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:12.221370 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:12.221401 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:12.225626 (+ 4225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:12.230314 (+ 4688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:12.230323 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:12.230339 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:12.230926 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:12.231461 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:12.231466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:12.231470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:12.231724 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:12.232788 (+ 1064us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:12.232797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:12.233189 (+ 392us) client_negotiation.cc:769] Sending connection context
0808 13:13:12.233488 (+ 299us) client_negotiation.cc:240] Negotiation successful
0808 13:13:12.234055 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21939,"thread_start_us":21759,"threads_started":1}
I20260808 13:13:12.237876 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:12.186060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55903 (local address 127.31.59.190:44719)
0808 13:13:12.212031 (+ 25971us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:12.212047 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:12.212089 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:13:12.212494 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:12.212509 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:12.213009 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:12.213494 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:12.215565 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.217200 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.218820 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.219829 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.220950 (+ 1121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:12.225977 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:12.225989 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:12.226008 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:12.226147 (+ 139us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:12.230127 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:12.231103 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:12.231112 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:12.231121 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:12.231328 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:12.231864 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:12.231870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:12.231873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:12.232515 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:12.232705 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:12.236286 (+ 3581us) server_negotiation.cc:299] Negotiation successful
0808 13:13:12.236680 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25748,"thread_start_us":25265,"threads_started":1}
I20260808 13:13:12.241137 13447 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:12.241494 13447 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:12.242349 13447 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
W20260808 13:13:12.245743 13447 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:44719 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:13:12.245927 13447 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:12.246171 13447 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:12.247362 13447 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:12.247578 13447 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
W20260808 13:13:12.248605 13447 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:13:12.585521 13451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:12.585819 13451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:12.585976 13451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:12.593837 13451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:12.593962 13451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:12.594144 13451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:12.604612 13451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:44843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=37079
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:12.606169 13451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:12.607431 13451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:12.621085 13467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:12.621078 13465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:12.625782 13464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:12.626937 13451 server_base.cc:1034] running on GCE node
I20260808 13:13:12.627611 13451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:12.628656 13451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:12.629863 13451 hybrid_clock.cc:648] HybridClock initialized: now 1786194792629799 us; error 58 us; skew 500 ppm
I20260808 13:13:12.630182 13451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:12.633510 13451 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:12.635154 13451 webserver.cc:460] Webserver started at http://127.31.59.130:37079/ using document root <none> and password file <none>
I20260808 13:13:12.635840 13451 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:12.635968 13451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:12.675305 13451 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.007s sys 0.013s
I20260808 13:13:12.680822 13477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.682897 13451 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 13:13:12.683097 13451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "8f3a4115850549bd86ab419786124287"
format_stamp: "Formatted at 2026-08-08 13:13:09 on dist-test-slave-49h3"
server_key: "e2ddb234e14ae58d2fdb229ffd18e41a"
server_key_iv: "b3c16aae6ad0c939e921a99b47f9ab3e"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:12.684590 13451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:12.703157 13451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:12.708284 13451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:12.708648 13451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:12.709946 13451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:12.711845 13451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:12.711961 13451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.712080 13451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:12.712163 13451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:12.810092 13451 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44843
I20260808 13:13:12.810150 13599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44843 every 8 connection(s)
I20260808 13:13:12.811726 13451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:12.821805 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13451
I20260808 13:13:12.822615 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.148:49399/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:46033
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:44719
--webserver_port=39631
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
Aug 08 13:13:12 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:12.853920 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:12.815327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.130:49707)
0808 13:13:12.830269 (+ 14942us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:12.830334 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:12.832733 (+ 2399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:12.834934 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:12.834946 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:12.835575 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:12.836366 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:12.836392 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.837923 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.837928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:12.839757 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:12.839772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.841004 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.841010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:12.841264 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:12.842081 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:12.842116 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:12.844613 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:12.848768 (+ 4155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:12.848778 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:12.848796 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:12.849421 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:12.849949 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:12.849954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:12.849958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:12.850240 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:12.851028 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:12.851036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:12.851494 (+ 458us) client_negotiation.cc:769] Sending connection context
0808 13:13:12.851837 (+ 343us) client_negotiation.cc:240] Negotiation successful
0808 13:13:12.852425 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14588,"thread_start_us":18042,"threads_started":1}
I20260808 13:13:12.855270 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:12.815964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:49707 (local address 127.31.59.190:44719)
0808 13:13:12.829568 (+ 13604us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:12.829575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:12.830805 (+ 1230us) server_negotiation.cc:407] Connection header received
0808 13:13:12.834170 (+ 3365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:12.834176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:12.834271 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:12.834359 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:12.836573 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.837713 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.839919 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:12.840509 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:12.840631 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:12.844858 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:12.844865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:12.844871 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:12.844917 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:12.848559 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:12.849619 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:12.849626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:12.849629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:12.849804 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:12.850387 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:12.850392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:12.850396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:12.850785 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:12.850949 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:12.854647 (+ 3698us) server_negotiation.cc:299] Negotiation successful
0808 13:13:12.854900 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13436,"thread_start_us":13623,"threads_started":1}
I20260808 13:13:12.856938 13600 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:12.857255 13600 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:12.858177 13600 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:12.860303 13236 ts_manager.cc:194] Registered new tserver with Master: 8f3a4115850549bd86ab419786124287 (127.31.59.130:44843)
I20260808 13:13:12.862529 13236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:49707
W20260808 13:13:13.244400 13604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:13.244696 13604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:13.244858 13604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:13:13.248764 13447 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:13.249191 13447 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:13.250547 13236 ts_manager.cc:194] Registered new tserver with Master: bf207803a986452d934e68a6acb95124 (127.31.59.129:45443)
I20260808 13:13:13.251694 13236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:55903
W20260808 13:13:13.254720 13604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:13.254889 13604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:13.255089 13604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:13.267012 13604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.148:49399/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:46033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=39631
--tserver_master_addrs=127.31.59.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:13.268409 13604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:13.269801 13604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 08 13:13:13 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
W20260808 13:13:13.298646 13612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:13.298870 13613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:13.315501 13615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:13.316447 13604 server_base.cc:1034] running on GCE node
I20260808 13:13:13.317132 13604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:13.320667 13604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:13.328449 13604 hybrid_clock.cc:648] HybridClock initialized: now 1786194793328136 us; error 283 us; skew 500 ppm
I20260808 13:13:13.329632 13604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:13 dist-test-slave-49h3 krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:13.336331 13604 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:13.350019 13604 webserver.cc:460] Webserver started at http://127.31.59.131:39631/ using document root <none> and password file <none>
I20260808 13:13:13.350806 13604 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:13.350968 13604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:13:13 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:13.415409 13604 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.012s sys 0.000s
I20260808 13:13:13.426103 13624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:13.435289 13604 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.006s sys 0.001s
I20260808 13:13:13.435526 13604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "a9e2980d3fab400a9ea9287f4b619010"
format_stamp: "Formatted at 2026-08-08 13:13:10 on dist-test-slave-49h3"
server_key: "8f5859773e097af9bac33c8c37f35d26"
server_key_iv: "f2e530d3d5ad706309c16a6e63204632"
server_key_version: "kuduclusterkey@0"
I20260808 13:13:13.437024 13604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:13.498859 13604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:13.503855 13604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:13.504220 13604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:13.511682 13604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:13.519914 13604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:13.520121 13604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:13.520284 13604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:13.520393 13604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:13:13 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for HTTP/127.31.59.148@KRBTEST.COM
I20260808 13:13:13.867381 13600 heartbeater.cc:503] Master 127.31.59.190:44719 was elected leader, sending a full tablet report...
I20260808 13:13:14.085323 13604 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:46033
I20260808 13:13:14.085356 13745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:46033 every 8 connection(s)
I20260808 13:13:14.086949 13604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:14.095263 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13604
Aug 08 13:13:14 dist-test-slave-49h3 krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:14.130206 13751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.106458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.31.59.190:44719)
0808 13:13:14.107334 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.107338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.107351 (+ 13us) server_negotiation.cc:407] Connection header received
0808 13:13:14.107385 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.107388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.107437 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.107528 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.109152 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.110391 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.112479 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.113219 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.113349 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.121080 (+ 7731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.121104 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.121109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.121161 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.124839 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.125952 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.125957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.125960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.126131 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.126717 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.126723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.126726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.127688 (+ 962us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.128010 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.129564 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.129822 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":664,"threads_started":1}
I20260808 13:13:14.134584 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.090546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:38073 (local address 127.31.59.190:44719)
0808 13:13:14.106497 (+ 15951us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.106503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.106523 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:13:14.106591 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.106596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.106712 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.106843 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.109234 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.110423 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.112130 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.112812 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.112924 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.126228 (+ 13304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.126237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.126242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.126295 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.129958 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.131251 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.131257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.131260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.131390 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.131964 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.131970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.131974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.132337 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.132517 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.133979 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.134211 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":15820,"thread_start_us":15674,"threads_started":1}
I20260808 13:13:14.135766 13749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.091148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44719 (local address 127.31.59.131:38073)
0808 13:13:14.102778 (+ 11630us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.102851 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.105617 (+ 2766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.107233 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.107247 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.108052 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:14.108960 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.108989 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.110596 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.110600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.111945 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.111961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.113264 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.113269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.121618 (+ 8349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.122742 (+ 1124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:14.122784 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:14.125899 (+ 3115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:14.130140 (+ 4241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:14.130151 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:14.130172 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:14.130815 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:14.131538 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:14.131542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:14.131545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:14.131828 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:14.132619 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:14.132628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:14.133099 (+ 471us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.133432 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.134033 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11217,"thread_start_us":10974,"threads_started":1}
I20260808 13:13:14.137477 13746 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44719
I20260808 13:13:14.137828 13746 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:14.138721 13746 heartbeater.cc:511] Master 127.31.59.190:44719 requested a full tablet report, sending...
I20260808 13:13:14.140295 13236 ts_manager.cc:194] Registered new tserver with Master: a9e2980d3fab400a9ea9287f4b619010 (127.31.59.131:46033)
I20260808 13:13:14.141093 13236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:38073
I20260808 13:13:14.146108 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:13:14.163733 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.149210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.31.59.190:44719)
0808 13:13:14.149420 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.149426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.149889 (+ 463us) server_negotiation.cc:407] Connection header received
0808 13:13:14.150036 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.150042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.150119 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.150219 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.151711 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.152827 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.154268 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.154933 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.155058 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.157480 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.157488 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.157493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.157533 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.160983 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.161725 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.161731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.161735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.161900 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.162245 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.162250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.162254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.162615 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.162876 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.163144 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.163384 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:13:14.168860 13236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:13:14.176230 13236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:13:14.217319 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.191300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.31.59.131:46033)
0808 13:13:14.208590 (+ 17290us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.208600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.208634 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:13:14.208749 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.208757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.209027 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.209243 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:14.210336 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.211953 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.214663 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.216085 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.216189 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.216460 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.216801 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.216965 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17103,"thread_start_us":17595,"threads_started":1}
I20260808 13:13:14.217363 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.190605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46033 (local address 127.0.0.1:59220)
0808 13:13:14.205409 (+ 14804us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.205468 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.205664 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.209620 (+ 3956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.209628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.209660 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:14.210098 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.210113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.212130 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.212134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.214499 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.214512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.216310 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.216320 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.216531 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.216603 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.216719 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.216918 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14519,"mutex_wait_us":29,"thread_start_us":18601,"threads_started":1}
I20260808 13:13:14.218703 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.190621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39458)
0808 13:13:14.209492 (+ 18871us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.209511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.209622 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.210520 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.210524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.210548 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:14.210939 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.210954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.213718 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.213724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.215866 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.215882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.218053 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.218060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.218229 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.218260 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.218333 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.218428 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18726,"mutex_wait_us":3,"thread_start_us":14357,"threads_started":1}
I20260808 13:13:14.219452 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.190641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.31.59.129:45443)
0808 13:13:14.208668 (+ 18027us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.208675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.209754 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 13:13:14.209859 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.209866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.210089 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.210266 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:14.211158 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.213577 (+ 2419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.216041 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.217558 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.217680 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.217905 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.218925 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.219096 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17871,"thread_start_us":18057,"threads_started":1}
I20260808 13:13:14.221125 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.191179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44843 (local address 127.0.0.1:38236)
0808 13:13:14.213572 (+ 22393us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.213593 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.213716 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.214289 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.214294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.214325 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:14.214772 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.214785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.216093 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.216098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.218207 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.218219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.220443 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.220447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.220564 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.220586 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.220681 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.220815 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22245,"thread_start_us":22027,"threads_started":1}
I20260808 13:13:14.221236 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.191475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.31.59.130:44843)
0808 13:13:14.212610 (+ 21135us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.212621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.213746 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 13:13:14.213854 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.213861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.214089 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.214293 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:14.214939 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.215925 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.218354 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.220001 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.220126 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.220329 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.220755 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.220921 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20931,"thread_start_us":21143,"threads_started":1}
I20260808 13:13:14.221908 13676 tablet_service.cc:1467] Processing CreateTablet for tablet e898188ee2dd47e2861414e6706a6587 (DEFAULT_TABLE table=test-table [id=61aad749c8b64c109afa2c8792ba7428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:14.223305 13676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e898188ee2dd47e2861414e6706a6587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:14.222991 13376 tablet_service.cc:1467] Processing CreateTablet for tablet e898188ee2dd47e2861414e6706a6587 (DEFAULT_TABLE table=test-table [id=61aad749c8b64c109afa2c8792ba7428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:14.224393 13376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e898188ee2dd47e2861414e6706a6587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:14.224893 13527 tablet_service.cc:1467] Processing CreateTablet for tablet e898188ee2dd47e2861414e6706a6587 (DEFAULT_TABLE table=test-table [id=61aad749c8b64c109afa2c8792ba7428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:14.226256 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e898188ee2dd47e2861414e6706a6587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:14.235342 13774 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Bootstrap starting.
I20260808 13:13:14.238544 13774 tablet_bootstrap.cc:654] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:14.239862 13774 log.cc:824] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:14.240803 13775 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Bootstrap starting.
I20260808 13:13:14.242502 13776 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Bootstrap starting.
I20260808 13:13:14.244251 13774 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: No bootstrap required, opened a new log
I20260808 13:13:14.244498 13774 ts_tablet_manager.cc:1397] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:13:14.245754 13775 tablet_bootstrap.cc:654] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:14.245966 13776 tablet_bootstrap.cc:654] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:14.247045 13775 log.cc:824] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:14.247262 13776 log.cc:824] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:14.250209 13775 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: No bootstrap required, opened a new log
I20260808 13:13:14.250221 13776 tablet_bootstrap.cc:492] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: No bootstrap required, opened a new log
I20260808 13:13:14.250476 13776 ts_tablet_manager.cc:1397] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:13:14.250476 13775 ts_tablet_manager.cc:1397] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260808 13:13:14.252210 13774 raft_consensus.cc:348] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.252631 13774 raft_consensus.cc:374] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:14.252722 13774 raft_consensus.cc:729] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e2980d3fab400a9ea9287f4b619010, State: Initialized, Role: FOLLOWER
I20260808 13:13:14.253326 13774 consensus_queue.cc:260] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.254457 13746 heartbeater.cc:503] Master 127.31.59.190:44719 was elected leader, sending a full tablet report...
I20260808 13:13:14.254972 13447 heartbeater.cc:503] Master 127.31.59.190:44719 was elected leader, sending a full tablet report...
I20260808 13:13:14.256388 13774 ts_tablet_manager.cc:1428] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 13:13:14.260727 13775 raft_consensus.cc:348] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.261171 13775 raft_consensus.cc:374] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:14.261276 13775 raft_consensus.cc:729] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3a4115850549bd86ab419786124287, State: Initialized, Role: FOLLOWER
I20260808 13:13:14.261390 13776 raft_consensus.cc:348] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.261798 13776 raft_consensus.cc:374] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:14.261895 13776 raft_consensus.cc:729] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf207803a986452d934e68a6acb95124, State: Initialized, Role: FOLLOWER
I20260808 13:13:14.261938 13775 consensus_queue.cc:260] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.262457 13776 consensus_queue.cc:260] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.263674 13775 ts_tablet_manager.cc:1428] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260808 13:13:14.264039 13776 ts_tablet_manager.cc:1428] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260808 13:13:14.295513 13782 raft_consensus.cc:482] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:13:14.295812 13782 raft_consensus.cc:504] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.297549 13782 leader_election.cc:288] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9e2980d3fab400a9ea9287f4b619010 (127.31.59.131:46033), 8f3a4115850549bd86ab419786124287 (127.31.59.130:44843)
I20260808 13:13:14.304123 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.298059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55883 (local address 127.31.59.131:46033)
0808 13:13:14.298261 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.298268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.299005 (+ 737us) server_negotiation.cc:407] Connection header received
0808 13:13:14.299191 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.299195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.299266 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.299339 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:14.299959 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.300693 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.302485 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.303239 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.303389 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.303467 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.303766 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.303889 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:13:14.304224 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.298243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:49225 (local address 127.31.59.130:44843)
0808 13:13:14.298465 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.298472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.299097 (+ 625us) server_negotiation.cc:407] Connection header received
0808 13:13:14.299293 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.299297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.299356 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.299431 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:14.300088 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.300775 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.302570 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.303335 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.303480 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.303550 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.303861 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.303985 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:13:14.304266 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.298063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:46033 (local address 127.31.59.129:55883)
0808 13:13:14.298873 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.298893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.299033 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.299371 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.299374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.299406 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:14.299807 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.299816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.300821 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.300826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.302326 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.302341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.303439 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.303445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.303613 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.303637 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.303714 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.303848 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":20,"thread_start_us":409,"threads_started":1}
I20260808 13:13:14.304363 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.298063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44843 (local address 127.31.59.129:49225)
0808 13:13:14.298994 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.299012 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:14.299148 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:14.299463 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:14.299467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:14.299573 (+ 106us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:14.299919 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.299928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.300916 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.300921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.302423 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:14.302435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.303551 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.303557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:14.303704 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.303728 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:13:14.303811 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:13:14.303927 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":95,"spinlock_wait_cycles":35200,"thread_start_us":501,"threads_started":1}
I20260808 13:13:14.305469 13554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e898188ee2dd47e2861414e6706a6587" candidate_uuid: "bf207803a986452d934e68a6acb95124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3a4115850549bd86ab419786124287" is_pre_election: true
I20260808 13:13:14.305475 13699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e898188ee2dd47e2861414e6706a6587" candidate_uuid: "bf207803a986452d934e68a6acb95124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e2980d3fab400a9ea9287f4b619010" is_pre_election: true
I20260808 13:13:14.306092 13699 raft_consensus.cc:2454] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf207803a986452d934e68a6acb95124 in term 0.
I20260808 13:13:14.306092 13554 raft_consensus.cc:2454] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf207803a986452d934e68a6acb95124 in term 0.
I20260808 13:13:14.306764 13329 leader_election.cc:302] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f3a4115850549bd86ab419786124287, bf207803a986452d934e68a6acb95124; no voters:
I20260808 13:13:14.307119 13782 raft_consensus.cc:2781] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:13:14.307286 13782 raft_consensus.cc:482] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:13:14.307385 13782 raft_consensus.cc:3037] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:14.309556 13782 raft_consensus.cc:504] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.310101 13782 leader_election.cc:288] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [CANDIDATE]: Term 1 election: Requested vote from peers a9e2980d3fab400a9ea9287f4b619010 (127.31.59.131:46033), 8f3a4115850549bd86ab419786124287 (127.31.59.130:44843)
I20260808 13:13:14.310563 13699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e898188ee2dd47e2861414e6706a6587" candidate_uuid: "bf207803a986452d934e68a6acb95124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e2980d3fab400a9ea9287f4b619010"
I20260808 13:13:14.310577 13554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e898188ee2dd47e2861414e6706a6587" candidate_uuid: "bf207803a986452d934e68a6acb95124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3a4115850549bd86ab419786124287"
I20260808 13:13:14.310725 13699 raft_consensus.cc:3037] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:14.310765 13554 raft_consensus.cc:3037] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:14.313235 13699 raft_consensus.cc:2454] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf207803a986452d934e68a6acb95124 in term 1.
I20260808 13:13:14.313441 13554 raft_consensus.cc:2454] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf207803a986452d934e68a6acb95124 in term 1.
I20260808 13:13:14.313630 13328 leader_election.cc:302] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9e2980d3fab400a9ea9287f4b619010, bf207803a986452d934e68a6acb95124; no voters:
I20260808 13:13:14.313966 13782 raft_consensus.cc:2781] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:14.314375 13782 raft_consensus.cc:686] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 1 LEADER]: Becoming Leader. State: Replica: bf207803a986452d934e68a6acb95124, State: Running, Role: LEADER
I20260808 13:13:14.314817 13782 consensus_queue.cc:237] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } }
I20260808 13:13:14.319303 13236 catalog_manager.cc:5495] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf207803a986452d934e68a6acb95124 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "bf207803a986452d934e68a6acb95124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf207803a986452d934e68a6acb95124" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 } health_report { overall_health: UNKNOWN } } }
W20260808 13:13:14.334913 13601 tablet.cc:2369] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:13:14.340667 13747 tablet.cc:2369] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:13:14.354549 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.349202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.31.59.129:45443)
0808 13:13:14.349435 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.349441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.349484 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:13:14.349714 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.349718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.349814 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.349947 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:13:14.350349 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.351081 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.352410 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.352810 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.352941 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.353426 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:13:14.353976 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:13:14.354109 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.354211 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.354297 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:13:14.362392 13699 raft_consensus.cc:1261] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 1 FOLLOWER]: Refusing update from remote peer bf207803a986452d934e68a6acb95124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:14.362388 13554 raft_consensus.cc:1261] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 1 FOLLOWER]: Refusing update from remote peer bf207803a986452d934e68a6acb95124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:14.363314 13785 consensus_queue.cc:1035] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3a4115850549bd86ab419786124287" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:14.363648 13782 consensus_queue.cc:1035] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9e2980d3fab400a9ea9287f4b619010" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:14.375337 13790 mvcc.cc:205] Tried to move back new op lower bound from 7316253877698879488 to 7316253877516922880. Current Snapshot: MvccSnapshot[applied={T|T < 7316253877698879488}]
I20260808 13:13:14.376154 13789 mvcc.cc:205] Tried to move back new op lower bound from 7316253877698879488 to 7316253877516922880. Current Snapshot: MvccSnapshot[applied={T|T < 7316253877698879488}]
I20260808 13:13:14.388489 13236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:14.388728 13236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:14.392202 13236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af5e4bf0eae4b0799d07b69b26829f7: Sending DeleteTablet for 3 replicas of tablet e898188ee2dd47e2861414e6706a6587
I20260808 13:13:14.393334 13376 tablet_service.cc:1514] Processing DeleteTablet for tablet e898188ee2dd47e2861414e6706a6587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:39458
I20260808 13:13:14.393332 13676 tablet_service.cc:1514] Processing DeleteTablet for tablet e898188ee2dd47e2861414e6706a6587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:59220
I20260808 13:13:14.393541 13527 tablet_service.cc:1514] Processing DeleteTablet for tablet e898188ee2dd47e2861414e6706a6587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:38236
I20260808 13:13:14.405838 13800 tablet_replica.cc:330] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: stopping tablet replica
I20260808 13:13:14.406240 13799 tablet_replica.cc:330] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: stopping tablet replica
I20260808 13:13:14.406418 13800 raft_consensus.cc:2229] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:14.406826 13799 raft_consensus.cc:2229] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:13:14.406975 13800 raft_consensus.cc:2258] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:14.407204 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13275
I20260808 13:13:14.407697 13799 raft_consensus.cc:2258] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:14.407799 13801 tablet_replica.cc:330] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: stopping tablet replica
I20260808 13:13:14.408183 13801 raft_consensus.cc:2229] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:14.408694 13801 raft_consensus.cc:2258] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:14.410998 13800 ts_tablet_manager.cc:1905] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:14.412581 13799 ts_tablet_manager.cc:1905] T e898188ee2dd47e2861414e6706a6587 P bf207803a986452d934e68a6acb95124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:14.413887 13801 ts_tablet_manager.cc:1905] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:14.417924 13800 ts_tablet_manager.cc:1918] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:14.418088 13800 log.cc:1196] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/e898188ee2dd47e2861414e6706a6587
I20260808 13:13:14.418653 13800 ts_tablet_manager.cc:1939] T e898188ee2dd47e2861414e6706a6587 P a9e2980d3fab400a9ea9287f4b619010: Deleting consensus metadata
I20260808 13:13:14.420599 13219 catalog_manager.cc:4841] TS a9e2980d3fab400a9ea9287f4b619010 (127.31.59.131:46033): tablet e898188ee2dd47e2861414e6706a6587 (table test-table [id=61aad749c8b64c109afa2c8792ba7428]) successfully deleted
I20260808 13:13:14.423293 13801 ts_tablet_manager.cc:1918] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:14.423460 13801 log.cc:1196] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/e898188ee2dd47e2861414e6706a6587
I20260808 13:13:14.423991 13801 ts_tablet_manager.cc:1939] T e898188ee2dd47e2861414e6706a6587 P 8f3a4115850549bd86ab419786124287: Deleting consensus metadata
I20260808 13:13:14.425801 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.409746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.31.59.129:45443)
0808 13:13:14.409917 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.409922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.409941 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:13:14.410000 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.410005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.410083 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.410200 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.411778 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.412966 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.414765 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.415466 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.415593 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.419008 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.419026 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.419043 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.419103 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.422679 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.423506 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.423516 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.423519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.423692 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.424151 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.424156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.424159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.424529 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.424816 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.425177 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.425417 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:13:14.426080 13220 catalog_manager.cc:4841] TS 8f3a4115850549bd86ab419786124287 (127.31.59.130:44843): tablet e898188ee2dd47e2861414e6706a6587 (table test-table [id=61aad749c8b64c109afa2c8792ba7428]) successfully deleted
I20260808 13:13:14.426577 13442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:14.524015 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13275
I20260808 13:13:14.543066 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13451
W20260808 13:13:14.543179 13218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:45443 (error 108)
I20260808 13:13:14.545205 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.544717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39488)
0808 13:13:14.544905 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.545029 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 13:13:14.545461 13218 catalog_manager.cc:4584] TS bf207803a986452d934e68a6acb95124 (127.31.59.129:45443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e898188ee2dd47e2861414e6706a6587: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
I20260808 13:13:14.556306 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.544005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.31.59.130:44843)
0808 13:13:14.544195 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.544201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.544227 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:13:14.544446 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.544450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.544527 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.544634 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.546225 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.546961 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.548212 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.548625 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.548717 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.551041 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.551057 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.551071 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.551111 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.553970 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.554573 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.554580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.554582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.554713 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.555000 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.555003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.555005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.555356 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.555597 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.555782 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.555946 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:13:14.557017 13595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:14.599138 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.598557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39498)
0808 13:13:14.598805 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.598939 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:13:14.600186 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.599784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39504)
0808 13:13:14.599939 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.600026 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 13:13:14.646978 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13451
I20260808 13:13:14.660696 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.660247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39512)
0808 13:13:14.660447 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.660538 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:13:14.661808 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.661440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39520)
0808 13:13:14.661610 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.661682 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 13:13:14.665441 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13604
I20260808 13:13:14.677850 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.666166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.31.59.131:46033)
0808 13:13:14.666337 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:14.666341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:14.666359 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:13:14.666510 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:14.666514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:14.666586 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:14.666665 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:14.667860 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.668586 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.669957 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:14.670380 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:14.670470 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:14.672786 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:14.672798 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:14.672814 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:14.672839 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:14.675503 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.676118 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.676125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.676127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.676275 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:14.676555 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:14.676558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:14.676560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:14.676888 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:14.677111 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:14.677326 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 13:13:14.677606 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:13:14.678515 13741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:14.742475 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.741995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39522)
0808 13:13:14.742211 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.742309 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 13:13:14.743712 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:14.743292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45443 (local address 127.0.0.1:39538)
0808 13:13:14.743521 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:14.743598 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 13:13:14.768368 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13604
I20260808 13:13:14.784605 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13199
I20260808 13:13:14.785898 13268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:14.902917 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13199
I20260808 13:13:14.929916 31982 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 13:13:15.274621 31982 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 13:13:15.274854 31982 mini_ranger.cc:66] Stopping Ranger...
I20260808 13:13:15.619323 31982 mini_ranger.cc:68] Stopped Ranger
2026-08-08 13:13:15.619 UTC [12187] LOG: received smart shutdown request
2026-08-08 13:13:15.622 UTC [12187] LOG: background worker "logical replication launcher" (PID 12196) exited with exit code 1
2026-08-08 13:13:15.622 UTC [12191] LOG: shutting down
2026-08-08 13:13:15.771 UTC [12187] LOG: database system is shut down
2026-08-08T13:13:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51132 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:13:16 dist-test-slave-49h3 krb5kdc[13816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:13:16 dist-test-slave-49h3 krb5kdc[13816](info): set up 2 sockets
Aug 08 13:13:16 dist-test-slave-49h3 krb5kdc[13816](info): commencing operation
krb5kdc: starting...
W20260808 13:13:16.981918 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:13:17 dist-test-slave-49h3 krb5kdc[13816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:13:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:13:17.256736 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42413
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:17.590668 13830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:17.590983 13830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:17.591131 13830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:17.598619 13830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:17.598753 13830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:17.598832 13830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:17.598910 13830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:17.598973 13830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:17.609086 13830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:17.610483 13830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:17.611791 13830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:17.623210 13841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:17.623231 13836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:17.623322 13839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:17.624009 13830 server_base.cc:1034] running on GCE node
I20260808 13:13:17.624713 13830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:17.626106 13830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:17.627295 13830 hybrid_clock.cc:648] HybridClock initialized: now 1786194797627252 us; error 38 us; skew 500 ppm
I20260808 13:13:17.627619 13830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:17 dist-test-slave-49h3 krb5kdc[13816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:17.635805 13830 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:17.637392 13830 webserver.cc:460] Webserver started at http://127.31.59.190:43351/ using document root <none> and password file <none>
I20260808 13:13:17.638113 13830 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:17.638231 13830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:17.638528 13830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:17.641669 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "62cf10dda9444e058a9c9930a921450a"
format_stamp: "Formatted at 2026-08-08 13:13:17 on dist-test-slave-49h3"
server_key: "69e51c36b1f331a305de6899150e3f62"
server_key_iv: "defe2c3016740751cecb24f45c153f4c"
server_key_version: "encryptionkey@0"
I20260808 13:13:17.642501 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "62cf10dda9444e058a9c9930a921450a"
format_stamp: "Formatted at 2026-08-08 13:13:17 on dist-test-slave-49h3"
server_key: "69e51c36b1f331a305de6899150e3f62"
server_key_iv: "defe2c3016740751cecb24f45c153f4c"
server_key_version: "encryptionkey@0"
I20260808 13:13:17.647687 13830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:13:17.651036 13848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:17.652276 13830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:13:17.652431 13830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "62cf10dda9444e058a9c9930a921450a"
format_stamp: "Formatted at 2026-08-08 13:13:17 on dist-test-slave-49h3"
server_key: "69e51c36b1f331a305de6899150e3f62"
server_key_iv: "defe2c3016740751cecb24f45c153f4c"
server_key_version: "encryptionkey@0"
I20260808 13:13:17.652606 13830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:17.664870 13830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:17.668545 13830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:17.668814 13830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:17.686655 13830 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:42413
I20260808 13:13:17.686658 13904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:42413 every 8 connection(s)
I20260808 13:13:17.688071 13830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:17.690644 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13830
I20260808 13:13:17.690862 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:13:17.691495 31982 external_mini_cluster.cc:1428] Setting key 43cf361c9bd91b892ff442b33f241548
I20260808 13:13:17.692076 13905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:17.698669 13905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:13:17 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.59.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.189@KRBTEST.COM" created.
I20260808 13:13:17.721174 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.701887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.31.59.190:42413)
0808 13:13:17.703784 (+ 1897us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:17.703795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:17.703858 (+ 63us) server_negotiation.cc:407] Connection header received
0808 13:13:17.705478 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:17.705487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:17.705762 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:17.705947 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:17.707295 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:17.708524 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:17.709976 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:17.710524 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:17.710689 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:17.714339 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:17.714348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:17.714361 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:17.714391 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:17.717115 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:17.717823 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:17.717830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:17.717833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:17.718008 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:17.718408 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:17.718412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:17.718414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:17.718914 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:17.719190 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:17.719739 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 13:13:17.720178 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":172,"server-negotiator.queue_time_us":1531,"spinlock_wait_cycles":17152,"thread_start_us":1306,"threads_started":1}
I20260808 13:13:17.721223 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.701886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42413 (local address 127.0.0.1:44872)
0808 13:13:17.702782 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.702832 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:17.705046 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:17.705997 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:17.706004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:17.706751 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:17.707095 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:17.707112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:17.708677 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:17.708681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:17.709844 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:17.709858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:17.710751 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:17.710759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:17.710931 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:17.711572 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:17.711594 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:17.714087 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:17.717243 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:17.717254 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:17.717267 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:17.717696 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:17.718120 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:17.718126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:17.718130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:17.718300 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:17.719043 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:17.719051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:17.719311 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 13:13:17.719519 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 13:13:17.720025 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":382,"threads_started":1}
I20260808 13:13:17.724027 13905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:13:17.725095 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.724620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59076)
0808 13:13:17.724829 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.724960 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 13:13:17.725333 13852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
I20260808 13:13:17.726953 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.726550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59090)
0808 13:13:17.726726 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.726808 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:13:17.727155 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:37181: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
I20260808 13:13:17.750869 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:37181
--webserver_interface=127.31.59.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:17.776520 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } attempt: 1
I20260808 13:13:17.777794 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.777293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59102)
0808 13:13:17.777536 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.777647 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:13:17.778790 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.778429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59116)
0808 13:13:17.778589 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.778657 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:13:17.778980 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:37181: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
I20260808 13:13:17.847213 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } attempt: 2
I20260808 13:13:17.848367 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.847880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59126)
0808 13:13:17.848119 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.848211 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:13:17.849437 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.849074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59136)
0808 13:13:17.849221 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.849310 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:13:17.849642 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:37181: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
I20260808 13:13:17.940882 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } attempt: 3
I20260808 13:13:17.942306 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.941785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59150)
0808 13:13:17.942078 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.942164 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260808 13:13:17.944670 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:17.944182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59154)
0808 13:13:17.944382 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:17.944495 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 13:13:17.944977 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:37181: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
I20260808 13:13:18.088248 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } attempt: 4
I20260808 13:13:18.089563 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.088951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59166)
0808 13:13:18.089270 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.089378 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260808 13:13:18.090572 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.090206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59176)
0808 13:13:18.090357 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.090450 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:13:18.090763 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:37181: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
W20260808 13:13:18.093926 13912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:18.094233 13912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:18.094368 13912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:18.102062 13912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:18.102191 13912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:18.102270 13912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:18.102348 13912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:18.102409 13912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:18.112735 13912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:37181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:18.114161 13912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:18.115466 13912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:18.127151 13918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:18.127211 13917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:18.127434 13920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:18.128134 13912 server_base.cc:1034] running on GCE node
I20260808 13:13:18.128906 13912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:18.130328 13912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:18.131524 13912 hybrid_clock.cc:648] HybridClock initialized: now 1786194798131483 us; error 27 us; skew 500 ppm
I20260808 13:13:18.131863 13912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:18.135067 13912 init.cc:377] Logged in from keytab as kudu/127.31.59.189@KRBTEST.COM (short username kudu)
I20260808 13:13:18.136497 13912 webserver.cc:460] Webserver started at http://127.31.59.189:45771/ using document root <none> and password file <none>
I20260808 13:13:18.137199 13912 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:18.137310 13912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:18.137657 13912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:18.140651 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data/instance:
uuid: "8c0bb9626f7742b4af88199773eed232"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "3c38ed0eb3189deaf79f86ca1edcb4c2"
server_key_iv: "72b70f074c7774c52aa1ae72b4b2c05c"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.141500 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal/instance:
uuid: "8c0bb9626f7742b4af88199773eed232"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "3c38ed0eb3189deaf79f86ca1edcb4c2"
server_key_iv: "72b70f074c7774c52aa1ae72b4b2c05c"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.146718 13912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:13:18.150308 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:18.151619 13912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:13:18.151762 13912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "8c0bb9626f7742b4af88199773eed232"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "3c38ed0eb3189deaf79f86ca1edcb4c2"
server_key_iv: "72b70f074c7774c52aa1ae72b4b2c05c"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.151922 13912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:18.175002 13912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:18.179255 13912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:18.179495 13912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:18.197541 13912 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.189:37181
I20260808 13:13:18.197587 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.189:37181 every 8 connection(s)
I20260808 13:13:18.198901 13912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
I20260808 13:13:18.202951 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:18.208153 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13912
I20260808 13:13:18.208348 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-1/wal/instance
I20260808 13:13:18.208778 31982 external_mini_cluster.cc:1428] Setting key 1612c7249932b7c0ddb5ace034f69ee8
I20260808 13:13:18.209848 13985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:18.230756 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.212564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.31.59.190:42413)
0808 13:13:18.212796 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.212804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.213997 (+ 1193us) server_negotiation.cc:407] Connection header received
0808 13:13:18.215907 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.215913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.215995 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.216117 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.217762 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.218654 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.220287 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.220844 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.220944 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.224994 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.225000 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.225003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.225038 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.227664 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.228481 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.228484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.228487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.228599 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.229080 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.229083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.229085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.229367 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.229511 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.230334 (+ 823us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.230518 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:13:18.232205 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.212955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42413 (local address 127.0.0.1:44884)
0808 13:13:18.213803 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.213865 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.215643 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.216398 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.216406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.216950 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.217570 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.217598 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.218827 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.218831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.220111 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.220127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.221084 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.221091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.221369 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.222074 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.222109 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.224741 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.227810 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.227819 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.227835 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.228378 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.228690 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.228695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.228698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.228981 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.229506 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.229513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.229952 (+ 439us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.230248 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.230830 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":365,"threads_started":1}
I20260808 13:13:18.234102 13985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.59.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.188@KRBTEST.COM" created.
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:13:18.249146 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.234932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59182)
0808 13:13:18.235164 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.235184 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.235304 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.236288 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.236298 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.236770 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.236968 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.236981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.238399 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.238403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.239303 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.239318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.240278 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.240283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.240441 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.241005 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.241020 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.243152 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.246899 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.246903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.246906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.247270 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.247621 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.247624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.247626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.247774 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.248415 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.248418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.248548 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.248654 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.248873 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 13:13:18.249488 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.235180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.31.59.189:37181)
0808 13:13:18.235869 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.235875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.235900 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:13:18.235943 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.235949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.236124 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.236230 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.237116 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.238275 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.239440 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.240135 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.240273 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.243369 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.243383 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.243400 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.243440 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.246776 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.247390 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.247398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.247400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.247531 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.247864 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.247868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.247871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.248303 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.248514 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.248821 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.249064 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":396,"threads_started":1}
I20260808 13:13:18.251868 13985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:13:18.253031 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.252472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34246)
0808 13:13:18.252707 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.252844 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 13:13:18.253314 13930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.254956 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.254556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34260)
0808 13:13:18.254712 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.254812 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 13:13:18.255167 13985 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
I20260808 13:13:18.272364 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:35609
--webserver_interface=127.31.59.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:18.304529 13985 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 1
I20260808 13:13:18.305852 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.305279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34266)
0808 13:13:18.305523 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.305652 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 13:13:18.306854 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.306478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34274)
0808 13:13:18.306609 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.306710 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 13:13:18.307049 13985 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.375294 13985 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 2
I20260808 13:13:18.376554 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.376005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34276)
0808 13:13:18.376236 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.376359 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 13:13:18.377563 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.377164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34284)
0808 13:13:18.377300 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.377384 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 13:13:18.377774 13985 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.390033 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } attempt: 5
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:13:18.403985 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.390998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.31.59.189:37181)
0808 13:13:18.391151 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.391155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.391174 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:13:18.391226 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.391229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.391311 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.391391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.392371 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.393157 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.394808 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.395242 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.395340 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.398968 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.398973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.398976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.399007 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.401555 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.402285 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.402289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.402292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.402402 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.402775 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.402778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.402780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.403043 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.403149 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.403579 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.403748 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:13:18.404127 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.390736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59184)
0808 13:13:18.390955 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.390978 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.391114 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.391453 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.391458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.391964 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.392214 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.392227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.393297 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.393303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.394671 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.394687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.395387 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.395393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.395587 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.396209 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.396231 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.398774 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.401682 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.401687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.401692 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.402186 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.402492 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.402497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.402500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.402670 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.403147 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.403152 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.403342 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.403496 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.403794 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:13:18.405222 13905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:13:18.406425 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.405952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34296)
0808 13:13:18.406144 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.406239 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 13:13:18.407423 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.407070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34304)
0808 13:13:18.407199 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.407288 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 13:13:18.407656 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.458863 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 1
I20260808 13:13:18.460045 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.459571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34312)
0808 13:13:18.459774 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.459870 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 13:13:18.461028 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.460717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34316)
0808 13:13:18.460846 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.460924 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 13:13:18.461236 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.469003 13985 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 3
I20260808 13:13:18.470218 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.469654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34320)
0808 13:13:18.469900 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.470013 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 13:13:18.471362 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.470860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34332)
0808 13:13:18.471111 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.471217 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260808 13:13:18.471589 13985 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
I20260808 13:13:18.529438 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 2
I20260808 13:13:18.530620 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.530152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34346)
0808 13:13:18.530349 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.530453 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 13:13:18.531560 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.531251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34358)
0808 13:13:18.531386 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.531464 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 13:13:18.531769 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
W20260808 13:13:18.612905 13993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:18.613185 13993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:18.613322 13993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:13:18.614812 13985 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 4
I20260808 13:13:18.616020 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.615444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34368)
0808 13:13:18.615705 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.615829 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:13:18.617084 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.616738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34382)
0808 13:13:18.616882 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.616959 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:13:18.617282 13985 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
W20260808 13:13:18.621198 13993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:18.621318 13993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:18.621397 13993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:18.621496 13993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:18.621572 13993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:18.631454 13993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42413,127.31.59.189:37181,127.31.59.188:35609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:35609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:18.632764 13993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:18.634078 13993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:13:18.637977 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 3
I20260808 13:13:18.639199 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.638620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34392)
0808 13:13:18.638900 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.639006 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 13:13:18.640520 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.639961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34398)
0808 13:13:18.640268 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.640365 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260808 13:13:18.640774 13905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:35609: Network error: Client connection negotiation failed: client connection to 127.31.59.188:35609: connect: Connection refused (error 111)
W20260808 13:13:18.645654 13998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:18.645699 14001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:18.645722 13999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:18.646867 13993 server_base.cc:1034] running on GCE node
I20260808 13:13:18.647580 13993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:18.648897 13993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:18.650094 13993 hybrid_clock.cc:648] HybridClock initialized: now 1786194798650040 us; error 45 us; skew 500 ppm
I20260808 13:13:18.650413 13993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:18.653319 13993 init.cc:377] Logged in from keytab as kudu/127.31.59.188@KRBTEST.COM (short username kudu)
I20260808 13:13:18.654718 13993 webserver.cc:460] Webserver started at http://127.31.59.188:37071/ using document root <none> and password file <none>
I20260808 13:13:18.655400 13993 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:18.655499 13993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:18.655802 13993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:18.658929 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data/instance:
uuid: "8cfbc515da984c7b91cc32ef88293d57"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "b9a780925ab94ac3bc5cdc9f03e34306"
server_key_iv: "26a5a5a81fba0fa4886f64dbcc36cb46"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.659723 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal/instance:
uuid: "8cfbc515da984c7b91cc32ef88293d57"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "b9a780925ab94ac3bc5cdc9f03e34306"
server_key_iv: "26a5a5a81fba0fa4886f64dbcc36cb46"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.664633 13993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:13:18.667881 14010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:18.669111 13993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:13:18.669263 13993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "8cfbc515da984c7b91cc32ef88293d57"
format_stamp: "Formatted at 2026-08-08 13:13:18 on dist-test-slave-49h3"
server_key: "b9a780925ab94ac3bc5cdc9f03e34306"
server_key_iv: "26a5a5a81fba0fa4886f64dbcc36cb46"
server_key_version: "encryptionkey@0"
I20260808 13:13:18.669435 13993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:18.686935 13993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:18.691107 13993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:18.691367 13993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:18.709203 13993 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.188:35609
I20260808 13:13:18.709235 14063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.188:35609 every 8 connection(s)
I20260808 13:13:18.710564 13993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
I20260808 13:13:18.714073 14064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:18.717996 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 13993
I20260808 13:13:18.718194 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194589892156-31982-0/minicluster-data/master-2/wal/instance
I20260808 13:13:18.718634 31982 external_mini_cluster.cc:1428] Setting key 938daab8709360e99676f6b529c9692c
I20260808 13:13:18.720916 14064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:13:18.739557 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.724978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.31.59.190:42413)
0808 13:13:18.725219 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.725226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.725799 (+ 573us) server_negotiation.cc:407] Connection header received
0808 13:13:18.725983 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.725989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.726065 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.726148 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.727349 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.728032 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.729609 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.730049 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.730169 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.733749 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.733754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.733757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.733790 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.736210 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.736979 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.736983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.736985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.737127 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.737579 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.737584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.737586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.737981 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.738102 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.738936 (+ 834us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.739172 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:13:18.740728 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.724913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42413 (local address 127.0.0.1:44886)
0808 13:13:18.725674 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.725712 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.725864 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.726248 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.726255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.726845 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.727183 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.727196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.728198 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.728204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.729475 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.729491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.730289 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.730298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.730475 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.731062 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.731091 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.733554 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.736344 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.736356 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.736369 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.736860 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.737256 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.737261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.737265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.737437 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.738097 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.738105 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.738503 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.738834 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.739328 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":18,"thread_start_us":365,"threads_started":1}
I20260808 13:13:18.740751 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.720442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.31.59.188:35609)
0808 13:13:18.721247 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.721258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.721299 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:13:18.723111 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.723135 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.723531 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.723842 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.725594 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.727026 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.728483 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.729155 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.729298 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.734277 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.734288 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.734304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.734341 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.737114 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.737952 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.737958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.737960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.738140 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.738504 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.738508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.738511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.738950 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.739230 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.739575 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.739814 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":595,"thread_start_us":387,"threads_started":1}
I20260808 13:13:18.742385 14064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:18.756529 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.743190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.31.59.189:37181)
0808 13:13:18.743388 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.743393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.743564 (+ 171us) server_negotiation.cc:407] Connection header received
0808 13:13:18.743722 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.743727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.743810 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.743899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.744594 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.745582 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.746799 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.747209 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.747306 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.750508 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.750515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.750520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.750563 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.753713 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.754399 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.754404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.754406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.754550 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.754940 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.754943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.754969 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.755320 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.755458 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.755943 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.756172 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:13:18.756573 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.743284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59188)
0808 13:13:18.743485 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.743499 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.743609 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.743954 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.743959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.744310 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.744453 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.744461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.745713 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.745717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.746650 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.746664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.747347 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.747353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.747561 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.748116 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.748133 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.750311 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.753861 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.753866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.753869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.754268 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.754658 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.754663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.754667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.754847 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.755452 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.755458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.755650 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.755816 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.756170 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 13:13:18.757638 14064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:13:18.760558 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.743891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.31.59.190:42413)
0808 13:13:18.744678 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.744683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.744944 (+ 261us) server_negotiation.cc:407] Connection header received
0808 13:13:18.745122 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.745125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.745181 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.745277 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.746566 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.747418 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.748745 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.749337 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.749432 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.754411 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.754417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.754420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.754453 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.757715 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.758501 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.758507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.758511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.758681 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.759152 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.759156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.759158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.759441 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.759686 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.759992 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.760230 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671}
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:13:18.771063 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.758335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34416)
0808 13:13:18.758490 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.758507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.758593 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.758946 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.758950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.759363 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.759513 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.759524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.760786 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.760791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.761895 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.761907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.762709 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.762714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.762955 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.763514 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.763529 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.765709 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.768551 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.768556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.768560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.769058 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.769395 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.769400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.769404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.769587 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.770078 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.770083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.770229 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.770357 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.770663 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 13:13:18.771080 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.758370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.31.59.188:35609)
0808 13:13:18.758541 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.758547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.758563 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:13:18.758686 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.758691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.758794 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.758894 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.759659 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.760637 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.762010 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.762574 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.762691 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.765893 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.765898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.765901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.765930 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.768425 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.769162 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.769165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.769168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.769291 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.769682 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.769685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.769687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.769956 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.770068 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.770476 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.770822 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":69}
I20260808 13:13:18.776310 14064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: Bootstrap starting.
I20260808 13:13:18.777043 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.762331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.31.59.189:37181)
0808 13:13:18.762484 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.762487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.762526 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:13:18.762716 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.762720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.762797 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.762896 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.764186 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.765008 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.766549 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.766970 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.767078 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.771583 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.771587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.771590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.771621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.774548 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.775200 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.775206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.775209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.775357 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.775700 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.775704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.775706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.775974 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.776256 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.776508 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.776738 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:13:18.779873 14064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:18.781008 14064 log.cc:824] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:18.781677 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:13:18.783684 14064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: No bootstrap required, opened a new log
I20260808 13:13:18.793994 14064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.794647 14064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:18.794826 14064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfbc515da984c7b91cc32ef88293d57, State: Initialized, Role: FOLLOWER
I20260808 13:13:18.795754 14064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.804003 14064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:18.806676 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.786841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.31.59.190:42413)
0808 13:13:18.787018 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.787023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.787805 (+ 782us) server_negotiation.cc:407] Connection header received
0808 13:13:18.788147 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.788154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.788246 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.788326 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.791138 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.792316 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.794851 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.795580 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.795680 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.799604 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.799609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.799612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.799653 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.803144 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.804043 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.804048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.804052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.804239 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.804722 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.804728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.804731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.805088 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.805347 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.806025 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.806298 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:13:18.807621 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.786450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.31.59.189:37181)
0808 13:13:18.786661 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.786666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.788004 (+ 1338us) server_negotiation.cc:407] Connection header received
0808 13:13:18.788257 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.788263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.788343 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.788411 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.791043 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.792158 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.794400 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.795003 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.795100 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.799956 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.799964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.800079 (+ 115us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.800135 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.803581 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.804548 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.804553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.804556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.804750 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.805192 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.805198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.805201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.805646 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.805917 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.806980 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.807240 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:13:18.808791 14077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:18.809304 14077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:18.817690 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.786859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.31.59.188:35609)
0808 13:13:18.787028 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.787034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.799666 (+ 12632us) server_negotiation.cc:407] Connection header received
0808 13:13:18.799852 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.799857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.799937 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.800027 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.802709 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.803840 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.806009 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.806701 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.806818 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.810310 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.810315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.810318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.810350 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.813635 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.814461 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.814467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.814471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.814677 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.815130 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.815134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.815137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.815548 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.815792 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.817009 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.817247 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 13:13:18.818204 13905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 4
W20260808 13:13:18.821563 14092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:13:18.821774 14092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:13:18.815953 13993 master_runner.cc:186] Error getting master registration for 127.31.59.190:42413: OK, instance_id { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" instance_seqno: 1786194797681504 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:13:18.827183 13993 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194797, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:13:18.833866 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.819491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.31.59.188:35609)
0808 13:13:18.819728 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.819733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.819748 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:13:18.819795 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.819802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.819920 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.820018 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.820940 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.822457 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.824140 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.824820 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.824928 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.828236 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.828244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.828248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.828285 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.831301 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.832071 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.832074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.832077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.832192 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.832579 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.832582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.832584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.832867 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.832972 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.833400 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.833615 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260808 13:13:18.834035 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.818739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34430)
0808 13:13:18.819077 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.819105 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.819207 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.820062 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.820067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.820538 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.820742 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.820766 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.822633 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.822639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.823973 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.823989 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.824965 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.824971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.825147 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.825786 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.825803 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.828019 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.831432 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.831437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.831441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.831962 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.832294 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.832299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.832303 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.832479 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.832989 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.832995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.833172 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.833334 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.833645 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260808 13:13:18.838883 13905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: Bootstrap starting.
I20260808 13:13:18.842279 13905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:18.843467 13905 log.cc:824] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:18.846098 13905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: No bootstrap required, opened a new log
I20260808 13:13:18.854071 13905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.854382 13905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:18.854494 13905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cf10dda9444e058a9c9930a921450a, State: Initialized, Role: FOLLOWER
I20260808 13:13:18.855043 13905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.856017 14096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:18.856495 14096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:18.857138 13905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: configured and running, proceeding with master startup.
W20260808 13:13:18.872315 14108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:13:18.872542 14108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:13:18.866858 13830 master_runner.cc:186] Error getting master registration for 127.31.59.189:37181: OK, instance_id { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" instance_seqno: 1786194798192160 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:13:18.875587 13830 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:13:18.916477 13985 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } attempt: 5
W20260808 13:13:18.921674 13854 tablet.cc:2369] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:13:18.922014 14077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:13:18.922348 14077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.924034 14077 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62cf10dda9444e058a9c9930a921450a (127.31.59.190:42413), 8c0bb9626f7742b4af88199773eed232 (127.31.59.189:37181)
Aug 08 13:13:18 dist-test-slave-49h3 krb5kdc[13816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194798, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:13:18.925177 13876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cfbc515da984c7b91cc32ef88293d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cf10dda9444e058a9c9930a921450a" is_pre_election: true
I20260808 13:13:18.925832 13876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cfbc515da984c7b91cc32ef88293d57 in term 0.
I20260808 13:13:18.926736 14011 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cf10dda9444e058a9c9930a921450a, 8cfbc515da984c7b91cc32ef88293d57; no voters:
I20260808 13:13:18.927881 14077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:13:18.928047 14077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:13:18.928174 14077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:18.935040 14077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.936367 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.917094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.31.59.188:35609)
0808 13:13:18.917304 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:18.917310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:18.917368 (+ 58us) server_negotiation.cc:407] Connection header received
0808 13:13:18.917587 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:18.917591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:18.917653 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:18.917737 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:18.918525 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.919273 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.920444 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.920924 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.921074 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.926498 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:18.926506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:18.926511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:18.926555 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:18.932191 (+ 5636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.933147 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.933152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.933155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.933315 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:18.933777 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:18.933781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:18.933784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:18.934182 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:18.934330 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:18.935707 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0808 13:13:18.936001 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 13:13:18.936877 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:18.917044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:35609 (local address 127.0.0.1:34440)
0808 13:13:18.917292 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:18.917313 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:18.917430 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:18.917780 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:18.917784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:18.918215 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:18.918385 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.918394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.919426 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.919432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.920313 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:18.920324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:18.921071 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:18.921077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:18.921308 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:18.923131 (+ 1823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:18.923154 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:18.926266 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:18.932420 (+ 6154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.932426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.932430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.932997 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.933461 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:18.933466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:18.933469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:18.933637 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:18.934393 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:18.934397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:18.934566 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 13:13:18.934707 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 13:13:18.935200 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 13:13:18.938380 13876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cfbc515da984c7b91cc32ef88293d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cf10dda9444e058a9c9930a921450a"
I20260808 13:13:18.938650 13876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:18.942281 13876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cfbc515da984c7b91cc32ef88293d57 in term 1.
I20260808 13:13:18.942906 14011 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cf10dda9444e058a9c9930a921450a, 8cfbc515da984c7b91cc32ef88293d57; no voters:
W20260808 13:13:18.943533 14017 tablet.cc:2369] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:13:18.943602 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: Bootstrap starting.
I20260808 13:13:18.947688 14077 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [CANDIDATE]: Term 1 election: Requested vote from peers 62cf10dda9444e058a9c9930a921450a (127.31.59.190:42413), 8c0bb9626f7742b4af88199773eed232 (127.31.59.189:37181)
I20260808 13:13:18.947988 13985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:18.948657 14077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:18.949462 13985 log.cc:824] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:18.949339 14077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [term 1 LEADER]: Becoming Leader. State: Replica: 8cfbc515da984c7b91cc32ef88293d57, State: Running, Role: LEADER
I20260808 13:13:18.952322 14077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.958673 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: No bootstrap required, opened a new log
I20260808 13:13:18.970995 13985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.971513 13985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:18.971629 13985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0bb9626f7742b4af88199773eed232, State: Initialized, Role: FOLLOWER
I20260808 13:13:18.973028 14110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfbc515da984c7b91cc32ef88293d57. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:18.973661 14110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:18.978022 13985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } }
I20260808 13:13:18.986544 14113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:18.988858 14113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:18.994032 14114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:18.994514 14114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:18.995579 13985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:18.995649 13956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cfbc515da984c7b91cc32ef88293d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0bb9626f7742b4af88199773eed232"
I20260808 13:13:18.996166 13956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:18.995595 13957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cfbc515da984c7b91cc32ef88293d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0bb9626f7742b4af88199773eed232" is_pre_election: true
I20260808 13:13:19.001964 13956 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cfbc515da984c7b91cc32ef88293d57 in term 1.
I20260808 13:13:19.011400 13956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [term 1 FOLLOWER]: Refusing update from remote peer 8cfbc515da984c7b91cc32ef88293d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:19.011437 13876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [term 1 FOLLOWER]: Refusing update from remote peer 8cfbc515da984c7b91cc32ef88293d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:19.012351 14077 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:19.012702 14110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:19.056344 14096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfbc515da984c7b91cc32ef88293d57. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.056762 14096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:19.063884 14114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfbc515da984c7b91cc32ef88293d57. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.064218 14114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:19.070531 14122 mvcc.cc:205] Tried to move back new op lower bound from 7316253896719831040 to 7316253896518303744. Current Snapshot: MvccSnapshot[applied={T|T < 7316253896719831040}]
I20260808 13:13:19.074203 14114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.074482 14114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:19.079654 14110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.079972 14110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:19.080305 14110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.080543 14110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:19.080651 14113 catalog_manager.cc:1347] Generated new cluster ID: 3aa5964fc2924daaa20a4c899cf1bd61
I20260808 13:13:19.080749 14113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:19.085799 14096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cfbc515da984c7b91cc32ef88293d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cf10dda9444e058a9c9930a921450a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42413 } } peers { permanent_uuid: "8c0bb9626f7742b4af88199773eed232" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 37181 } } peers { permanent_uuid: "8cfbc515da984c7b91cc32ef88293d57" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 35609 } } }
I20260808 13:13:19.086079 14096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:13:19.116997 14134 catalog_manager.cc:1259] Loaded cluster ID: 3aa5964fc2924daaa20a4c899cf1bd61
I20260808 13:13:19.117197 14134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: loading cluster ID for follower catalog manager: success
W20260808 13:13:19.118562 14134 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:13:19.118784 14134 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:13:19.150467 14113 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:13:19.152374 14113 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:19.177145 14113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57: Generated new TSK 0
I20260808 13:13:19.181842 14113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:13:19.874806 14108 catalog_manager.cc:1259] Loaded cluster ID: 3aa5964fc2924daaa20a4c899cf1bd61
I20260808 13:13:19.874914 14108 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: loading cluster ID for follower catalog manager: success
I20260808 13:13:19.877372 14108 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: acquiring CA information for follower catalog manager: success
I20260808 13:13:19.879215 14108 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 62cf10dda9444e058a9c9930a921450a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:13:20.122255 14134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: acquiring CA information for follower catalog manager: success
I20260808 13:13:20.124166 14134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c0bb9626f7742b4af88199773eed232: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:13:20.334028 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13830
I20260808 13:13:20.335505 13899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:20.448154 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13830
I20260808 13:13:20.471848 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13912
I20260808 13:13:20.473014 13977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:20.581820 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13912
I20260808 13:13:20.591603 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:20.590809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42413 (local address 127.0.0.1:44914)
0808 13:13:20.591293 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:20.591421 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":271,"threads_started":1}
W20260808 13:13:20.591888 14011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.190:42413: connect: Connection refused (error 111)
I20260808 13:13:20.594583 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:20.594197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:42413 (local address 127.0.0.1:44926)
0808 13:13:20.594366 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:20.594455 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:13:20.594786 14011 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 -> Peer 62cf10dda9444e058a9c9930a921450a (127.31.59.190:42413): Couldn't send request to peer 62cf10dda9444e058a9c9930a921450a. Status: Network error: Client connection negotiation failed: client connection to 127.31.59.190:42413: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260808 13:13:20.605230 14015 connection.cc:535] client connection to 127.31.59.189:37181 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:13:20.605584 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 13993
I20260808 13:13:20.606479 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:20.606019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:37181 (local address 127.0.0.1:59202)
0808 13:13:20.606213 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:20.606312 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 13:13:20.606657 14059 generic_service.cc:196] Checking for leaks (request via RPC)
W20260808 13:13:20.606660 14015 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8cfbc515da984c7b91cc32ef88293d57 -> Peer 8c0bb9626f7742b4af88199773eed232 (127.31.59.189:37181): Couldn't send request to peer 8c0bb9626f7742b4af88199773eed232. Status: Network error: Client connection negotiation failed: client connection to 127.31.59.189:37181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:13:20.718577 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 13993
2026-08-08T13:13:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4830 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:13:20 dist-test-slave-49h3 krb5kdc[14148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:13:20 dist-test-slave-49h3 krb5kdc[14148](info): set up 2 sockets
Aug 08 13:13:20 dist-test-slave-49h3 krb5kdc[14148](info): commencing operation
krb5kdc: starting...
W20260808 13:13:21.806540 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:13:21 dist-test-slave-49h3 krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:13:22.097114 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45395
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:22.440291 14163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:22.440572 14163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:22.440668 14163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:22.447896 14163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:22.448006 14163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:22.448101 14163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:22.448174 14163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:22.448217 14163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:22.458034 14163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:22.459336 14163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:22.460644 14163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:22.472275 14168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:22.472235 14171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:22.472316 14169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:22.473371 14163 server_base.cc:1034] running on GCE node
I20260808 13:13:22.474150 14163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:22.475518 14163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:22.476733 14163 hybrid_clock.cc:648] HybridClock initialized: now 1786194802476674 us; error 30 us; skew 500 ppm
I20260808 13:13:22.477049 14163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:22 dist-test-slave-49h3 krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194802, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:22.479990 14163 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:22.481487 14163 webserver.cc:460] Webserver started at http://127.31.59.190:34659/ using document root <none> and password file <none>
I20260808 13:13:22.482146 14163 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:22.482257 14163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:22.482538 14163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:22.485507 14163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "f7e6b7ac1631481cbdebbd90dc298e1f"
format_stamp: "Formatted at 2026-08-08 13:13:22 on dist-test-slave-49h3"
server_key: "8b4661493f82d83528f33f01dc432f59"
server_key_iv: "84d03539920be806e5956942483a2153"
server_key_version: "encryptionkey@0"
I20260808 13:13:22.486258 14163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "f7e6b7ac1631481cbdebbd90dc298e1f"
format_stamp: "Formatted at 2026-08-08 13:13:22 on dist-test-slave-49h3"
server_key: "8b4661493f82d83528f33f01dc432f59"
server_key_iv: "84d03539920be806e5956942483a2153"
server_key_version: "encryptionkey@0"
I20260808 13:13:22.491190 14163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260808 13:13:22.494539 14180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:22.495795 14163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:13:22.495972 14163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "f7e6b7ac1631481cbdebbd90dc298e1f"
format_stamp: "Formatted at 2026-08-08 13:13:22 on dist-test-slave-49h3"
server_key: "8b4661493f82d83528f33f01dc432f59"
server_key_iv: "84d03539920be806e5956942483a2153"
server_key_version: "encryptionkey@0"
I20260808 13:13:22.496160 14163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:22.527618 14163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:22.531668 14163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:22.531962 14163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:22.550158 14163 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:45395
I20260808 13:13:22.550195 14235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:45395 every 8 connection(s)
I20260808 13:13:22.551509 14163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:22.554994 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14163
I20260808 13:13:22.555246 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:13:22.555460 14236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:22.556151 31982 external_mini_cluster.cc:1428] Setting key a16c4b6315a8f21f02d9152bf6690573
I20260808 13:13:22.569999 14236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f: Bootstrap starting.
Aug 08 13:13:22 dist-test-slave-49h3 krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:22.573191 14236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:22.574337 14236 log.cc:824] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:22.577195 14236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f: No bootstrap required, opened a new log
I20260808 13:13:22.579463 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:22.558006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.31.59.190:45395)
0808 13:13:22.558847 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:22.558859 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:22.558904 (+ 45us) server_negotiation.cc:407] Connection header received
0808 13:13:22.560521 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:22.560542 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:22.560956 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:22.561281 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:22.563113 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:22.564378 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:22.566286 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:22.566968 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:22.567100 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:22.571847 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:22.571855 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:22.571869 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:22.571914 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:22.575007 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:22.575777 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:22.575784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:22.575790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:22.575958 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:22.576329 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:22.576337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:22.576339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:22.576869 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:22.577161 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:22.577661 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 13:13:22.578144 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":397,"threads_started":1}
I20260808 13:13:22.583557 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:13:22.586617 14236 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } }
I20260808 13:13:22.586983 14236 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:22.587082 14236 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e6b7ac1631481cbdebbd90dc298e1f, State: Initialized, Role: FOLLOWER
I20260808 13:13:22.587662 14236 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } }
I20260808 13:13:22.587879 14236 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:22.588003 14236 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:22.588151 14236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:22.590386 14236 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } }
I20260808 13:13:22.590895 14236 leader_election.cc:302] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e6b7ac1631481cbdebbd90dc298e1f; no voters:
I20260808 13:13:22.591482 14236 leader_election.cc:288] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:13:22.591634 14241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:22.592308 14241 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [term 1 LEADER]: Becoming Leader. State: Replica: f7e6b7ac1631481cbdebbd90dc298e1f, State: Running, Role: LEADER
I20260808 13:13:22.592813 14241 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } }
I20260808 13:13:22.593498 14236 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:22.595405 14243 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e6b7ac1631481cbdebbd90dc298e1f. Latest consensus state: current_term: 1 leader_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } } }
I20260808 13:13:22.595858 14243 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:22.595377 14242 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e6b7ac1631481cbdebbd90dc298e1f" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45395 } } }
I20260808 13:13:22.596832 14242 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:22.598255 14248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:22.603700 14248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:22.616159 14248 catalog_manager.cc:1347] Generated new cluster ID: dd1e3ec84b1e4138a6f28314e9b83d02
I20260808 13:13:22.616269 14248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:22.635259 14248 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:13:22.637212 14248 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:22.647589 14248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e6b7ac1631481cbdebbd90dc298e1f: Generated new TSK 0
I20260808 13:13:22.648638 14248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:13:22.696075 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14163
I20260808 13:13:22.697212 14231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:22.800161 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14163
2026-08-08T13:13:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2075 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:13:22 dist-test-slave-49h3 krb5kdc[14266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:13:22 dist-test-slave-49h3 krb5kdc[14266](info): set up 2 sockets
Aug 08 13:13:22 dist-test-slave-49h3 krb5kdc[14266](info): commencing operation
krb5kdc: starting...
W20260808 13:13:23.894017 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:13:24 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:13:24.186975 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:24.523833 14281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:24.524135 14281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:24.524278 14281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:24.531898 14281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:24.532021 14281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:24.532084 14281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:24.532174 14281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:24.532251 14281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:24.542512 14281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:24.543880 14281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:24.545207 14281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:24.557204 14286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:24.557166 14287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:24.557101 14289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:24.557950 14281 server_base.cc:1034] running on GCE node
I20260808 13:13:24.558691 14281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:24.560045 14281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:24.561256 14281 hybrid_clock.cc:648] HybridClock initialized: now 1786194804561186 us; error 51 us; skew 500 ppm
I20260808 13:13:24.561645 14281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:24 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:24.564648 14281 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:24.566157 14281 webserver.cc:460] Webserver started at http://127.31.59.190:38411/ using document root <none> and password file <none>
I20260808 13:13:24.566836 14281 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:24.566958 14281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:24.567250 14281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:24.570286 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "2bd93266089843de990540ab2c7bfb47"
format_stamp: "Formatted at 2026-08-08 13:13:24 on dist-test-slave-49h3"
server_key: "e88d134b2fefed4e63e08c0382233d72"
server_key_iv: "ca86d73145cc12043d43304c50067486"
server_key_version: "encryptionkey@0"
I20260808 13:13:24.571041 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "2bd93266089843de990540ab2c7bfb47"
format_stamp: "Formatted at 2026-08-08 13:13:24 on dist-test-slave-49h3"
server_key: "e88d134b2fefed4e63e08c0382233d72"
server_key_iv: "ca86d73145cc12043d43304c50067486"
server_key_version: "encryptionkey@0"
I20260808 13:13:24.576103 14281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:13:24.579408 14296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:24.580621 14281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 13:13:24.580783 14281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2bd93266089843de990540ab2c7bfb47"
format_stamp: "Formatted at 2026-08-08 13:13:24 on dist-test-slave-49h3"
server_key: "e88d134b2fefed4e63e08c0382233d72"
server_key_iv: "ca86d73145cc12043d43304c50067486"
server_key_version: "encryptionkey@0"
I20260808 13:13:24.580971 14281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:24.628512 14281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:24.632944 14281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:24.633220 14281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:24.651096 14281 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40209
I20260808 13:13:24.651141 14354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40209 every 8 connection(s)
I20260808 13:13:24.652469 14281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:24.654255 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14281
I20260808 13:13:24.654457 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:13:24.655136 31982 external_mini_cluster.cc:1428] Setting key c2a7396105c5c76449caa629a8091758
I20260808 13:13:24.656381 14355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:13:24 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:24.670410 14355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap starting.
I20260808 13:13:24.674530 14355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:24.675796 14355 log.cc:824] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:24.679118 14355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: No bootstrap required, opened a new log
I20260808 13:13:24.679356 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:24.657182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.31.59.190:40209)
0808 13:13:24.658004 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:24.658019 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:24.658068 (+ 49us) server_negotiation.cc:407] Connection header received
0808 13:13:24.659854 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:24.659881 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:24.660378 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:24.660818 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:24.662161 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:24.664188 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:24.665566 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:24.666617 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:24.666817 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:24.671021 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:24.671031 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:24.671051 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:24.671097 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:24.674927 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:24.675644 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:24.675653 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:24.675661 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:24.675883 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:24.676311 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:24.676316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:24.676320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:24.676917 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:24.677197 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:24.677636 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 13:13:24.678064 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":330,"threads_started":1}
I20260808 13:13:24.686795 14355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:24.687121 14355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:24.687191 14355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Initialized, Role: FOLLOWER
I20260808 13:13:24.687662 14355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:24.687949 14355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:24.688089 14355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:24.688261 14355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:24.690433 14355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:24.690909 14355 leader_election.cc:302] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bd93266089843de990540ab2c7bfb47; no voters:
I20260808 13:13:24.691470 14355 leader_election.cc:288] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:13:24.691596 14360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:24.692247 14360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 LEADER]: Becoming Leader. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Running, Role: LEADER
I20260808 13:13:24.692797 14360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:24.693437 14355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:24.696463 14361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:24.697175 14361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:24.697022 14362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bd93266089843de990540ab2c7bfb47. Latest consensus state: current_term: 1 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:24.700774 14362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:24.706933 14368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:24.711597 14368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:24.719791 14368 catalog_manager.cc:1347] Generated new cluster ID: f17ef3f597c3447ca60456d59e7cd03e
I20260808 13:13:24.719894 14368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:24.737844 14368 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:13:24.739216 14368 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:24.745882 14368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Generated new TSK 0
I20260808 13:13:24.746699 14368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:13:24.812599 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:25.157334 14383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:25.157630 14383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:25.157738 14383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:25.164955 14383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:25.165067 14383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:25.165190 14383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:25.174947 14383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:25.176188 14383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:25.177438 14383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:25.190179 14391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:25.190227 14389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:25.190191 14388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:25.191318 14383 server_base.cc:1034] running on GCE node
I20260808 13:13:25.191870 14383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:25.192761 14383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:25.193967 14383 hybrid_clock.cc:648] HybridClock initialized: now 1786194805193913 us; error 48 us; skew 500 ppm
I20260808 13:13:25.194269 14383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:25 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194805, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:25.197469 14383 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:25.198889 14383 webserver.cc:460] Webserver started at http://127.31.59.129:44029/ using document root <none> and password file <none>
I20260808 13:13:25.199565 14383 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:25.199694 14383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:25.200043 14383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:25.203123 14383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "36cc49179e3e70e9a5c5df94dc0b70e3"
server_key_iv: "1da42c6df2be779c266f5aee53790e62"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.203964 14383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "36cc49179e3e70e9a5c5df94dc0b70e3"
server_key_iv: "1da42c6df2be779c266f5aee53790e62"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.209270 14383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:13:25.212718 14398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.214037 14383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:25.214190 14383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "36cc49179e3e70e9a5c5df94dc0b70e3"
server_key_iv: "1da42c6df2be779c266f5aee53790e62"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.214383 14383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:25.244261 14383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:25.248585 14383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:25.248869 14383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:25.249866 14383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:25.251252 14383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:25.251334 14383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.251458 14383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:25.251523 14383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.282786 14383 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44131
I20260808 13:13:25.282825 14520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44131 every 8 connection(s)
I20260808 13:13:25.284109 14383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:25.291607 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14383
I20260808 13:13:25.291817 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:13:25.292475 31982 external_mini_cluster.cc:1428] Setting key 1ce6633db4145ac38feff5bef6215ac9
Aug 08 13:13:25 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194805, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:25.303467 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:25.286611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:53711 (local address 127.31.59.190:40209)
0808 13:13:25.287028 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:25.287033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:25.288013 (+ 980us) server_negotiation.cc:407] Connection header received
0808 13:13:25.289846 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:25.289852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:25.289928 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:25.290025 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:25.291483 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.292235 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.293888 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.294332 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.294489 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:25.297961 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:25.297966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:25.297969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:25.298002 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:25.300429 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:25.301134 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:25.301138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:25.301140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:25.301260 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:25.301717 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:25.301721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:25.301723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:25.301997 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:25.302129 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:25.302862 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 13:13:25.303095 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":202,"threads_started":1}
I20260808 13:13:25.304450 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:25.287164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:53711)
0808 13:13:25.287848 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:25.287890 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:25.289588 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:25.290261 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:25.290269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:25.290756 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:25.291321 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:25.291339 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.292397 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.292401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:25.293745 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:25.293759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.294595 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.294603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:25.294834 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:25.295489 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:25.295512 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:25.297748 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:25.300564 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:25.300574 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:25.300591 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:25.301013 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:25.301361 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:25.301366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:25.301369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:25.301606 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:25.302120 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:25.302128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:25.302486 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 13:13:25.302724 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 13:13:25.303283 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":284,"threads_started":1}
I20260808 13:13:25.306075 14521 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:25.306368 14521 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:25.307144 14521 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:25.309341 14318 ts_manager.cc:194] Registered new tserver with Master: d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:25.311215 14318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:53711
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:13:25.354964 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:25.691236 14527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:25.691522 14527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:25.691663 14527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:25.699115 14527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:25.699229 14527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:25.699362 14527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:25.709200 14527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:25.710469 14527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:25.711724 14527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:25.724452 14533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:25.724514 14534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:25.724537 14536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:25.725382 14527 server_base.cc:1034] running on GCE node
I20260808 13:13:25.725994 14527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:25.726894 14527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:25.728104 14527 hybrid_clock.cc:648] HybridClock initialized: now 1786194805728052 us; error 44 us; skew 500 ppm
I20260808 13:13:25.728422 14527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:25 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194805, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:25.731554 14527 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:25.732908 14527 webserver.cc:460] Webserver started at http://127.31.59.130:41969/ using document root <none> and password file <none>
I20260808 13:13:25.733614 14527 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:25.733721 14527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:25.733965 14527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:25.736727 14527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "3da41418b7d671cbb5c8c6ed9498b9ad"
server_key_iv: "a50f041506d8bd60f7c4808d6e9adb20"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.737483 14527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "3da41418b7d671cbb5c8c6ed9498b9ad"
server_key_iv: "a50f041506d8bd60f7c4808d6e9adb20"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.742431 14527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:13:25.745710 14544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.747061 14527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:13:25.747202 14527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "3da41418b7d671cbb5c8c6ed9498b9ad"
server_key_iv: "a50f041506d8bd60f7c4808d6e9adb20"
server_key_version: "encryptionkey@0"
I20260808 13:13:25.747385 14527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:25.764863 14527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:25.769125 14527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:25.769397 14527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:25.770403 14527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:25.771813 14527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:25.771888 14527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.772010 14527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:25.772063 14527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:25.802080 14527 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:40803
I20260808 13:13:25.802131 14666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:40803 every 8 connection(s)
I20260808 13:13:25.803334 14527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:25.803823 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14527
I20260808 13:13:25.804034 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:13:25.804687 31982 external_mini_cluster.cc:1428] Setting key 178e3e329dfc5be19fe2ecc7beb29387
Aug 08 13:13:25 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194805, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:25.823964 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:25.806147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:55683 (local address 127.31.59.190:40209)
0808 13:13:25.806633 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:25.806638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:25.807761 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 13:13:25.809660 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:25.809666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:25.809743 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:25.809851 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:25.811310 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.812339 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.813896 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.814462 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.814581 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:25.818005 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:25.818011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:25.818015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:25.818056 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:25.821017 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:25.821741 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:25.821746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:25.821749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:25.821902 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:25.822301 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:25.822305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:25.822308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:25.822620 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:25.822758 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:25.823377 (+ 619us) server_negotiation.cc:299] Negotiation successful
0808 13:13:25.823597 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
I20260808 13:13:25.824666 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:25.806784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:55683)
0808 13:13:25.807589 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:25.807663 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:25.809374 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:25.810094 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:25.810102 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:25.810599 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:25.811141 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:25.811157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.812485 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.812489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:25.813762 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:25.813774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:25.814643 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:25.814647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:25.814849 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:25.815456 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:25.815492 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:25.817786 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:25.821143 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:25.821149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:25.821162 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:25.821622 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:25.821996 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:25.822000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:25.822002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:25.822172 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:25.822717 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:25.822723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:25.823013 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 13:13:25.823223 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 13:13:25.823596 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":312,"threads_started":1}
I20260808 13:13:25.826128 14667 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:25.826463 14667 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:25.827354 14667 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:25.828855 14318 ts_manager.cc:194] Registered new tserver with Master: 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803)
I20260808 13:13:25.829727 14318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:55683
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:13:25.870160 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:26.207383 14673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:26.207747 14673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:26.207896 14673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:26.215615 14673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:26.215768 14673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:26.215942 14673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:26.226337 14673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:26.227784 14673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:26.229133 14673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:26.242596 14679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:26.242519 14681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:26.242570 14683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:26.243729 14673 server_base.cc:1034] running on GCE node
I20260808 13:13:26.244321 14673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:26.245218 14673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:26.246413 14673 hybrid_clock.cc:648] HybridClock initialized: now 1786194806246347 us; error 59 us; skew 500 ppm
I20260808 13:13:26.246716 14673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:26 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194806, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:26.249723 14673 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:26.251261 14673 webserver.cc:460] Webserver started at http://127.31.59.131:43585/ using document root <none> and password file <none>
I20260808 13:13:26.251955 14673 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:26.252063 14673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:26.252316 14673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:26.255261 14673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "cf7dfd82989242e69fae1b05a9db04ee"
format_stamp: "Formatted at 2026-08-08 13:13:26 on dist-test-slave-49h3"
server_key: "52731499a70582b1ec18f6b6b2e012b5"
server_key_iv: "5fd5dcba462d40d63241b49ee7e78be1"
server_key_version: "encryptionkey@0"
I20260808 13:13:26.256016 14673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "cf7dfd82989242e69fae1b05a9db04ee"
format_stamp: "Formatted at 2026-08-08 13:13:26 on dist-test-slave-49h3"
server_key: "52731499a70582b1ec18f6b6b2e012b5"
server_key_iv: "5fd5dcba462d40d63241b49ee7e78be1"
server_key_version: "encryptionkey@0"
I20260808 13:13:26.261065 14673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:13:26.264509 14691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:26.265975 14673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:26.266147 14673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "cf7dfd82989242e69fae1b05a9db04ee"
format_stamp: "Formatted at 2026-08-08 13:13:26 on dist-test-slave-49h3"
server_key: "52731499a70582b1ec18f6b6b2e012b5"
server_key_iv: "5fd5dcba462d40d63241b49ee7e78be1"
server_key_version: "encryptionkey@0"
I20260808 13:13:26.266330 14673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:26.278129 14673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:26.281862 14673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:26.282132 14673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:26.283131 14673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:26.284471 14673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:26.284545 14673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:26.284662 14673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:26.284716 14673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:26.314318 14521 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:26.314349 14673 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:41413
I20260808 13:13:26.314368 14809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:41413 every 8 connection(s)
I20260808 13:13:26.315760 14673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:26.318113 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14673
I20260808 13:13:26.318303 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:13:26.318776 31982 external_mini_cluster.cc:1428] Setting key 78593eb38d2fa89bc632dc9c98ca389f
Aug 08 13:13:26 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194806, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:26.335719 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.318360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:40383 (local address 127.31.59.190:40209)
0808 13:13:26.319411 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.319417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.319818 (+ 401us) server_negotiation.cc:407] Connection header received
0808 13:13:26.322084 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.322089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.322182 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.322264 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:26.323722 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.324441 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.326042 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.326436 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.326546 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.330217 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:26.330223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:26.330226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:26.330260 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:26.332792 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.333652 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.333656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.333659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.333791 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.334241 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.334246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.334250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.334566 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:26.334687 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.335305 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.335476 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910}
I20260808 13:13:26.336687 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.318818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:40383)
0808 13:13:26.319668 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.319716 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.321802 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.322513 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.322521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.323010 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:26.323566 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.323583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.324597 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.324603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.325891 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.325904 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.326653 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.326661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.327026 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.327747 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:26.327769 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:26.330011 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:26.332921 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:26.332930 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:26.332944 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:26.333539 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:26.333897 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:26.333902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:26.333905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:26.334136 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:26.334670 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:26.334676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:26.335003 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.335225 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.335625 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":321,"threads_started":1}
I20260808 13:13:26.338073 14810 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:26.338380 14810 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:26.339063 14810 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:26.340420 14318 ts_manager.cc:194] Registered new tserver with Master: cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413)
I20260808 13:13:26.341192 14318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:40383
I20260808 13:13:26.346438 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:13:26.362794 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.350912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.31.59.190:40209)
0808 13:13:26.351117 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.351124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.351487 (+ 363us) server_negotiation.cc:407] Connection header received
0808 13:13:26.351705 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.351709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.351794 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.351875 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:26.353177 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.353949 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.355212 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.355621 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.355712 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.358022 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:26.358027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:26.358030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:26.358059 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:26.360429 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.361077 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.361080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.361082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.361205 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.361527 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.361530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.361532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.361866 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:26.362062 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.362300 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.362470 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:13:26.366724 14318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:13:26.373564 14318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:13:26.407706 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.385607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.31.59.130:40803)
0808 13:13:26.397979 (+ 12372us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.397990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.398820 (+ 830us) server_negotiation.cc:407] Connection header received
0808 13:13:26.399031 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.399038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.399298 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.399486 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:26.400607 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.402238 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.404937 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.406473 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.406610 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.406830 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.407186 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.407360 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12153,"thread_start_us":12019,"threads_started":1}
I20260808 13:13:26.408104 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.385392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:40803 (local address 127.0.0.1:41222)
0808 13:13:26.398468 (+ 13076us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.398520 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.398753 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.399780 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.399788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.399839 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:26.400394 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.400410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.402387 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.402392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.404759 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.404773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.406670 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.406677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.406885 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.406935 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.407129 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.407376 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12812,"mutex_wait_us":82,"spinlock_wait_cycles":3328,"thread_start_us":12666,"threads_started":1}
I20260808 13:13:26.408452 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.385906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.31.59.131:41413)
0808 13:13:26.398503 (+ 12597us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.398511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.398913 (+ 402us) server_negotiation.cc:407] Connection header received
0808 13:13:26.399030 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.399038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.399298 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.399486 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:26.401165 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.402739 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.405683 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.406724 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.406852 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.407074 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.407979 (+ 905us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.408143 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12432,"thread_start_us":12273,"threads_started":1}
I20260808 13:13:26.409595 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.399269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.31.59.129:44131)
0808 13:13:26.400575 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.400583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.401183 (+ 600us) server_negotiation.cc:407] Connection header received
0808 13:13:26.401284 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.401290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.401564 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.401756 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:26.402453 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.403545 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.405773 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.407139 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.407289 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.407490 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.409083 (+ 1593us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.409254 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":940,"threads_started":1}
I20260808 13:13:26.411715 14598 tablet_service.cc:1467] Processing CreateTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 (DEFAULT_TABLE table=test-table [id=1b49ae52ca884589bd7f858794f9b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:26.412398 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.385608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:41413 (local address 127.0.0.1:33958)
0808 13:13:26.398755 (+ 13147us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.398772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.398900 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.399780 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.399788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.399831 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:26.400936 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.400950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.402895 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.402900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.405524 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.405538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.407210 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.407217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.407391 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.407421 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.408638 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.408759 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13015,"mutex_wait_us":591,"spinlock_wait_cycles":1408,"thread_start_us":12933,"threads_started":1}
I20260808 13:13:26.412762 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.399157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50604)
0808 13:13:26.400934 (+ 1777us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.400950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.401067 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.401923 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.401927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.401952 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:26.402278 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.402286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.403685 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.403689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.405606 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.405622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.407646 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.407652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.408062 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.408100 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.409036 (+ 936us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.409164 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"mutex_wait_us":60,"thread_start_us":1444,"threads_started":1}
I20260808 13:13:26.413170 14598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5ae715d59c64aaea60c9fd8e3457166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:26.416071 14452 tablet_service.cc:1467] Processing CreateTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 (DEFAULT_TABLE table=test-table [id=1b49ae52ca884589bd7f858794f9b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:26.416512 14742 tablet_service.cc:1467] Processing CreateTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 (DEFAULT_TABLE table=test-table [id=1b49ae52ca884589bd7f858794f9b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:26.417300 14452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5ae715d59c64aaea60c9fd8e3457166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:26.418031 14742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5ae715d59c64aaea60c9fd8e3457166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:26.428454 14829 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Bootstrap starting.
I20260808 13:13:26.430966 14828 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Bootstrap starting.
I20260808 13:13:26.432309 14829 tablet_bootstrap.cc:654] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:26.432380 14830 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Bootstrap starting.
I20260808 13:13:26.433794 14829 log.cc:824] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:26.435218 14828 tablet_bootstrap.cc:654] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:26.435330 14830 tablet_bootstrap.cc:654] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:26.436408 14830 log.cc:824] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:26.436440 14828 log.cc:824] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:26.437544 14829 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: No bootstrap required, opened a new log
I20260808 13:13:26.437856 14829 ts_tablet_manager.cc:1397] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 13:13:26.444088 14830 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: No bootstrap required, opened a new log
I20260808 13:13:26.444376 14830 ts_tablet_manager.cc:1397] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260808 13:13:26.444834 14828 tablet_bootstrap.cc:492] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: No bootstrap required, opened a new log
I20260808 13:13:26.445101 14828 ts_tablet_manager.cc:1397] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260808 13:13:26.449051 14829 raft_consensus.cc:348] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.449532 14829 raft_consensus.cc:374] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:26.449632 14829 raft_consensus.cc:729] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f2df7ce6f943a0b921a1ebb512f380, State: Initialized, Role: FOLLOWER
I20260808 13:13:26.450285 14829 consensus_queue.cc:260] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.452044 14830 raft_consensus.cc:348] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.452369 14830 raft_consensus.cc:374] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:26.452461 14830 raft_consensus.cc:729] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7dfd82989242e69fae1b05a9db04ee, State: Initialized, Role: FOLLOWER
I20260808 13:13:26.452965 14830 consensus_queue.cc:260] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.454015 14810 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:26.455606 14828 raft_consensus.cc:348] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.455910 14830 ts_tablet_manager.cc:1428] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260808 13:13:26.456048 14828 raft_consensus.cc:374] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:26.456146 14828 raft_consensus.cc:729] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e77ad1d2154f2fbab93cd74c6a0078, State: Initialized, Role: FOLLOWER
I20260808 13:13:26.456717 14828 consensus_queue.cc:260] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.457903 14667 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:26.458549 14828 ts_tablet_manager.cc:1428] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 13:13:26.460026 14829 ts_tablet_manager.cc:1428] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Time spent starting tablet: real 0.022s user 0.010s sys 0.008s
I20260808 13:13:26.467979 14835 raft_consensus.cc:482] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:13:26.468233 14835 raft_consensus.cc:504] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.469861 14835 leader_election.cc:288] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803), d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:26.476257 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.470259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:40803 (local address 127.31.59.131:53111)
0808 13:13:26.470605 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.470625 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.470771 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.471114 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.471119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.471147 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:26.471322 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.471329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.472495 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.472499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.474410 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.474425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.475690 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.475694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.475832 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.475855 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.475924 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.476012 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":21}
I20260808 13:13:26.476442 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.470331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:53111 (local address 127.31.59.130:40803)
0808 13:13:26.470540 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.470547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.470743 (+ 196us) server_negotiation.cc:407] Connection header received
0808 13:13:26.470926 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.470930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.470997 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.471072 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:26.471448 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.472368 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.474548 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.475557 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.475713 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.475785 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.475963 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.476088 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:13:26.477010 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.470456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:57233 (local address 127.31.59.129:44131)
0808 13:13:26.470652 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.470659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.471107 (+ 448us) server_negotiation.cc:407] Connection header received
0808 13:13:26.471255 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.471259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.471316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.471376 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:26.471880 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.472793 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.474969 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.476094 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.476244 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.476308 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.476658 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.476789 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:13:26.477594 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.470314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.31.59.131:57233)
0808 13:13:26.471012 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:26.471025 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:26.471131 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:26.471414 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:26.471418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:26.471442 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:26.471745 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.471754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.472928 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.472934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.474836 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:26.474846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.476308 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.476314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:26.476479 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.476512 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:13:26.477169 (+ 657us) client_negotiation.cc:240] Negotiation successful
0808 13:13:26.477286 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":1,"thread_start_us":422,"threads_started":1}
I20260808 13:13:26.477437 14620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5ae715d59c64aaea60c9fd8e3457166" candidate_uuid: "cf7dfd82989242e69fae1b05a9db04ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" is_pre_election: true
I20260808 13:13:26.477967 14620 raft_consensus.cc:2454] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf7dfd82989242e69fae1b05a9db04ee in term 0.
I20260808 13:13:26.478490 14473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5ae715d59c64aaea60c9fd8e3457166" candidate_uuid: "cf7dfd82989242e69fae1b05a9db04ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" is_pre_election: true
I20260808 13:13:26.478603 14694 leader_election.cc:302] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11e77ad1d2154f2fbab93cd74c6a0078, cf7dfd82989242e69fae1b05a9db04ee; no voters:
I20260808 13:13:26.478952 14835 raft_consensus.cc:2781] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:13:26.478940 14473 raft_consensus.cc:2454] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf7dfd82989242e69fae1b05a9db04ee in term 0.
I20260808 13:13:26.479065 14835 raft_consensus.cc:482] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:13:26.479154 14835 raft_consensus.cc:3037] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:26.481374 14835 raft_consensus.cc:504] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.482021 14835 leader_election.cc:288] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [CANDIDATE]: Term 1 election: Requested vote from peers 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803), d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:26.482383 14620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5ae715d59c64aaea60c9fd8e3457166" candidate_uuid: "cf7dfd82989242e69fae1b05a9db04ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
I20260808 13:13:26.482463 14473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5ae715d59c64aaea60c9fd8e3457166" candidate_uuid: "cf7dfd82989242e69fae1b05a9db04ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
I20260808 13:13:26.482558 14620 raft_consensus.cc:3037] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:26.482652 14473 raft_consensus.cc:3037] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:26.485204 14620 raft_consensus.cc:2454] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf7dfd82989242e69fae1b05a9db04ee in term 1.
I20260808 13:13:26.485205 14473 raft_consensus.cc:2454] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf7dfd82989242e69fae1b05a9db04ee in term 1.
I20260808 13:13:26.485605 14692 leader_election.cc:302] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf7dfd82989242e69fae1b05a9db04ee, d0f2df7ce6f943a0b921a1ebb512f380; no voters:
I20260808 13:13:26.485916 14835 raft_consensus.cc:2781] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:26.486344 14835 raft_consensus.cc:686] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 1 LEADER]: Becoming Leader. State: Replica: cf7dfd82989242e69fae1b05a9db04ee, State: Running, Role: LEADER
I20260808 13:13:26.486779 14835 consensus_queue.cc:237] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } }
I20260808 13:13:26.491386 14314 catalog_manager.cc:5495] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee reported cstate change: term changed from 0 to 1, leader changed from <none> to cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131). New cstate: current_term: 1 leader_uuid: "cf7dfd82989242e69fae1b05a9db04ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } health_report { overall_health: UNKNOWN } } }
I20260808 13:13:26.516161 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.510595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.31.59.131:41413)
0808 13:13:26.510797 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.510802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.510826 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:13:26.511085 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.511088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.511170 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.511314 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:13:26.511688 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.512436 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.514031 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.514444 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.514555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.515073 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:13:26.515573 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:13:26.515708 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.515823 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.515910 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:13:26.524230 14473 raft_consensus.cc:1261] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Refusing update from remote peer cf7dfd82989242e69fae1b05a9db04ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:26.524230 14620 raft_consensus.cc:1261] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Refusing update from remote peer cf7dfd82989242e69fae1b05a9db04ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:26.525154 14835 consensus_queue.cc:1035] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:26.525478 14838 consensus_queue.cc:1035] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:13:26.537290 14522 tablet.cc:2369] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:13:26.554031 14318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:26.554301 14318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:26.558269 14318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Sending DeleteTablet for 3 replicas of tablet c5ae715d59c64aaea60c9fd8e3457166
I20260808 13:13:26.559414 14598 tablet_service.cc:1514] Processing DeleteTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:26 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:41222
I20260808 13:13:26.559595 14452 tablet_service.cc:1514] Processing DeleteTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:26 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:50604
I20260808 13:13:26.559840 14742 tablet_service.cc:1514] Processing DeleteTablet for tablet c5ae715d59c64aaea60c9fd8e3457166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:26 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33958
I20260808 13:13:26.560662 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14383
I20260808 13:13:26.571375 14851 tablet_replica.cc:330] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: stopping tablet replica
I20260808 13:13:26.571583 14854 tablet_replica.cc:330] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: stopping tablet replica
I20260808 13:13:26.571910 14851 raft_consensus.cc:2229] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:26.572511 14851 raft_consensus.cc:2258] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:26.572480 14854 raft_consensus.cc:2229] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:13:26.573334 14853 tablet_replica.cc:330] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: stopping tablet replica
I20260808 13:13:26.573698 14854 raft_consensus.cc:2258] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:26.573781 14853 raft_consensus.cc:2229] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:26.574762 14853 raft_consensus.cc:2258] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:26.576011 14854 ts_tablet_manager.cc:1905] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:26.576175 14851 ts_tablet_manager.cc:1905] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:26.579142 14853 ts_tablet_manager.cc:1905] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:26.583451 14854 ts_tablet_manager.cc:1918] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:26.583657 14854 log.cc:1196] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/c5ae715d59c64aaea60c9fd8e3457166
I20260808 13:13:26.583689 14851 ts_tablet_manager.cc:1918] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:26.583875 14851 log.cc:1196] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/c5ae715d59c64aaea60c9fd8e3457166
I20260808 13:13:26.584280 14854 ts_tablet_manager.cc:1939] T c5ae715d59c64aaea60c9fd8e3457166 P cf7dfd82989242e69fae1b05a9db04ee: Deleting consensus metadata
I20260808 13:13:26.584401 14851 ts_tablet_manager.cc:1939] T c5ae715d59c64aaea60c9fd8e3457166 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting consensus metadata
I20260808 13:13:26.586302 14299 catalog_manager.cc:4841] TS 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803): tablet c5ae715d59c64aaea60c9fd8e3457166 (table test-table [id=1b49ae52ca884589bd7f858794f9b33c]) successfully deleted
I20260808 13:13:26.586627 14297 catalog_manager.cc:4841] TS cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413): tablet c5ae715d59c64aaea60c9fd8e3457166 (table test-table [id=1b49ae52ca884589bd7f858794f9b33c]) successfully deleted
Aug 08 13:13:26 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:13:26.589579 14853 ts_tablet_manager.cc:1918] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:26.589776 14853 log.cc:1196] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/c5ae715d59c64aaea60c9fd8e3457166
I20260808 13:13:26.590273 14853 ts_tablet_manager.cc:1939] T c5ae715d59c64aaea60c9fd8e3457166 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting consensus metadata
I20260808 13:13:26.591836 14297 catalog_manager.cc:4841] TS d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131): tablet c5ae715d59c64aaea60c9fd8e3457166 (table test-table [id=1b49ae52ca884589bd7f858794f9b33c]) successfully deleted
I20260808 13:13:26.596732 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.561937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.31.59.129:44131)
0808 13:13:26.562125 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.562131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.570931 (+ 8800us) server_negotiation.cc:407] Connection header received
0808 13:13:26.571447 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.571453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.571548 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.571629 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:26.573802 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.575018 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.576991 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.577658 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.577775 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.589939 (+ 12164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:26.589978 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:26.589996 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:26.590045 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:26.593849 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.594564 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.594575 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.594578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.594766 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.595123 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.595128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.595131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.595574 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:26.595899 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.596112 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.596347 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:13:26.597429 14515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:26.689554 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14383
I20260808 13:13:26.710281 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14527
Aug 08 13:13:26 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:13:26.727018 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.711072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.31.59.130:40803)
0808 13:13:26.711265 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.711271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.711293 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:13:26.711403 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.711408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.711489 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.711599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:26.712815 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.714083 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.715361 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.715975 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.716103 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.720410 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:26.720431 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:26.720452 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:26.720502 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:26.724169 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.724821 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.724831 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.724835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.724990 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.725309 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.725314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.725318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.725886 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:26.726191 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.726408 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.726635 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:13:26.727804 14661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:26.816879 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14527
I20260808 13:13:26.835788 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14673
Aug 08 13:13:26 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:13:26.852527 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:26.836742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.31.59.131:41413)
0808 13:13:26.836952 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:26.836962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:26.836985 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:13:26.837050 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:26.837054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:26.837119 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:26.837206 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:26.838652 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.839545 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.840799 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:26.841256 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:26.841371 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:26.846249 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:26.846266 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:26.846281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:26.846322 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:26.849430 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.850343 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.850350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.850352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.850515 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:26.850897 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:26.850900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:26.850902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:26.851275 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:26.851524 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:26.851957 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 13:13:26.852163 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:13:26.853418 14805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:26.977344 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14673
I20260808 13:13:26.999266 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14281
I20260808 13:13:27.000445 14350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:27.113427 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14281
I20260808 13:13:27.135952 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--webserver_interface=127.31.59.190
--webserver_port=38411
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:27.471606 14860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:27.471920 14860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:27.472069 14860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:27.479465 14860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:27.479591 14860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:27.479671 14860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:27.479761 14860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:27.479820 14860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:27.489717 14860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:27.491024 14860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:27.492350 14860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:27.516961 14866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:27.516955 14869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:27.516991 14867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:27.522292 14860 server_base.cc:1034] running on GCE node
I20260808 13:13:27.523257 14860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:27.524923 14860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:27.526151 14860 hybrid_clock.cc:648] HybridClock initialized: now 1786194807526079 us; error 39 us; skew 500 ppm
I20260808 13:13:27.526548 14860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:27 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194807, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:27.530179 14860 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:27.531940 14860 webserver.cc:460] Webserver started at http://127.31.59.190:38411/ using document root <none> and password file <none>
I20260808 13:13:27.532706 14860 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:27.532835 14860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:27.538772 14860 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 13:13:27.541678 14878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:27.543028 14860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:27.543215 14860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2bd93266089843de990540ab2c7bfb47"
format_stamp: "Formatted at 2026-08-08 13:13:24 on dist-test-slave-49h3"
server_key: "e88d134b2fefed4e63e08c0382233d72"
server_key_iv: "ca86d73145cc12043d43304c50067486"
server_key_version: "encryptionkey@0"
I20260808 13:13:27.544271 14860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:27.567552 14860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:27.571969 14860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:27.572248 14860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:27.590762 14860 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40209
I20260808 13:13:27.590796 14934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40209 every 8 connection(s)
I20260808 13:13:27.592298 14860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:27.599483 14935 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:13:27.602545 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14860
I20260808 13:13:27.602936 14935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap starting.
I20260808 13:13:27.603364 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:44131
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=44029
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:27.623813 14935 log.cc:824] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:27.640264 14935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:13:27.640893 14935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap complete.
I20260808 13:13:27.654317 14935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:27.655454 14935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Initialized, Role: FOLLOWER
I20260808 13:13:27.656162 14935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:27.656445 14935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:27.656565 14935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:27.656752 14935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:13:27.662410 14935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:27.663043 14935 leader_election.cc:302] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bd93266089843de990540ab2c7bfb47; no voters:
I20260808 13:13:27.663739 14935 leader_election.cc:288] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:13:27.663883 14939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:13:27.665206 14939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 LEADER]: Becoming Leader. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Running, Role: LEADER
I20260808 13:13:27.665729 14939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:27.666013 14935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:27.668072 14940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:27.668439 14940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:27.672197 14941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bd93266089843de990540ab2c7bfb47. Latest consensus state: current_term: 2 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:27.672530 14941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:27.673821 14946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:27.681586 14946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:27.683024 14946 catalog_manager.cc:1259] Loaded cluster ID: f17ef3f597c3447ca60456d59e7cd03e
I20260808 13:13:27.683131 14946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:27.686348 14946 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 13:13:27.703922 14946 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 13:13:27.715762 14957 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:13:27.725747 14946 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:27.727699 14946 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Loaded TSK: 0
I20260808 13:13:27.729218 14946 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:13:27.979085 14937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:27.979403 14937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:27.979527 14937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:27.987089 14937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:27.987223 14937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:27.987399 14937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:27.997560 14937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:44131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=44029
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:27.998939 14937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:28.000268 14937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:28.014984 14964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.015161 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.015306 14967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:28.015995 14937 server_base.cc:1034] running on GCE node
I20260808 13:13:28.016685 14937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:28.017643 14937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:28.018841 14937 hybrid_clock.cc:648] HybridClock initialized: now 1786194808018789 us; error 47 us; skew 500 ppm
I20260808 13:13:28.019142 14937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:28 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:28.026718 14937 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:28.028326 14937 webserver.cc:460] Webserver started at http://127.31.59.129:44029/ using document root <none> and password file <none>
I20260808 13:13:28.029021 14937 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:28.029157 14937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:28.034672 14937 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 13:13:28.037343 14974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.038684 14937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:13:28.038846 14937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "36cc49179e3e70e9a5c5df94dc0b70e3"
server_key_iv: "1da42c6df2be779c266f5aee53790e62"
server_key_version: "encryptionkey@0"
I20260808 13:13:28.039817 14937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:28.056845 14937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:28.061581 14937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:28.061870 14937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:28.062845 14937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:28.064116 14937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:28.064208 14937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.064287 14937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:28.064342 14937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.098312 14937 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44131
I20260808 13:13:28.098361 15095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44131 every 8 connection(s)
I20260808 13:13:28.099773 14937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:28.102926 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 14937
I20260808 13:13:28.103618 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:40803
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=41969
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:28 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:28.127270 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:28.102839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:54749)
0808 13:13:28.106071 (+ 3232us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:28.106120 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:28.108344 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:28.109761 (+ 1417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:28.109771 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:28.110434 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:28.111166 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:28.111190 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.112943 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.112948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:28.114673 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:28.114687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.116003 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.116008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:28.116191 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:28.116886 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:28.116910 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:28.119341 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:28.123368 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:28.123374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:28.123385 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:28.123795 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:28.124277 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:28.124280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:28.124283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:28.124450 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:28.125336 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:28.125341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:28.125662 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 13:13:28.125887 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 13:13:28.126297 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2916,"thread_start_us":2715,"threads_started":1}
I20260808 13:13:28.127729 15098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:28.102735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54749 (local address 127.31.59.190:40209)
0808 13:13:28.103456 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:28.103474 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:28.106279 (+ 2805us) server_negotiation.cc:407] Connection header received
0808 13:13:28.108915 (+ 2636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:28.108928 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:28.109271 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:28.109595 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:28.111397 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.112781 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.114841 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.115786 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.115972 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:28.119644 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:28.119654 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:28.119672 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:28.119715 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:28.123229 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:28.123941 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:28.123949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:28.123957 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:28.124185 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:28.124584 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:28.124589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:28.124592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:28.125223 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:28.125407 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:28.126157 (+ 750us) server_negotiation.cc:299] Negotiation successful
0808 13:13:28.126619 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":290,"threads_started":1}
I20260808 13:13:28.130349 15096 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:28.130712 15096 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:28.131680 15096 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:28.133983 14898 ts_manager.cc:194] Registered new tserver with Master: d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:28.135401 14898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:54749
W20260808 13:13:28.444877 15100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:28.445174 15100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:28.445307 15100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:28.452726 15100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:28.452855 15100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:28.453028 15100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:28.462726 15100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=41969
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:28.464020 15100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:28.465330 15100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:28.478304 15109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.478525 15105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.478332 15107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:28.479190 15100 server_base.cc:1034] running on GCE node
I20260808 13:13:28.479717 15100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:28.480571 15100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:28.481757 15100 hybrid_clock.cc:648] HybridClock initialized: now 1786194808481704 us; error 44 us; skew 500 ppm
I20260808 13:13:28.482050 15100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:28 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:28.485052 15100 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:28.486501 15100 webserver.cc:460] Webserver started at http://127.31.59.130:41969/ using document root <none> and password file <none>
I20260808 13:13:28.487116 15100 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:28.487238 15100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:28.492399 15100 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 13:13:28.494993 15117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.496232 15100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:13:28.496388 15100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "3da41418b7d671cbb5c8c6ed9498b9ad"
server_key_iv: "a50f041506d8bd60f7c4808d6e9adb20"
server_key_version: "encryptionkey@0"
I20260808 13:13:28.497346 15100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:28.526743 15100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:28.530862 15100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:28.531136 15100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:28.532184 15100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:28.533615 15100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:28.533718 15100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.533826 15100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:28.533896 15100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.565733 15100 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:40803
I20260808 13:13:28.565814 15237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:40803 every 8 connection(s)
I20260808 13:13:28.567253 15100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:28.574888 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15100
I20260808 13:13:28.575495 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:41413
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=43585
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:13:28 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:28.587713 15098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:28.569729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:47971 (local address 127.31.59.190:40209)
0808 13:13:28.569921 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:28.569926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:28.571122 (+ 1196us) server_negotiation.cc:407] Connection header received
0808 13:13:28.572979 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:28.572985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:28.573066 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:28.573155 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:28.574769 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.575498 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.577677 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.578151 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.578250 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:28.582032 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:28.582036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:28.582039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:28.582074 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:28.584465 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:28.585187 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:28.585190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:28.585193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:28.585338 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:28.585798 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:28.585802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:28.585804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:28.586098 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:28.586235 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:28.587077 (+ 842us) server_negotiation.cc:299] Negotiation successful
0808 13:13:28.587337 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:13:28.588773 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:28.570257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:47971)
0808 13:13:28.570966 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:28.571003 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:28.572700 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:28.573411 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:28.573419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:28.573939 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:28.574570 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:28.574595 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.575675 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.575680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:28.577524 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:28.577554 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:28.578374 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:28.578381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:28.578654 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:28.579339 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:28.579366 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:28.581820 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:28.584599 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:28.584608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:28.584625 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:28.585063 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:28.585470 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:28.585476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:28.585480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:28.585678 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:28.586225 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:28.586233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:28.586655 (+ 422us) client_negotiation.cc:769] Sending connection context
0808 13:13:28.586940 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 13:13:28.587509 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":270,"threads_started":1}
I20260808 13:13:28.590397 15238 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:28.590737 15238 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:28.591634 15238 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:28.593300 14898 ts_manager.cc:194] Registered new tserver with Master: 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803)
I20260808 13:13:28.594450 14898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:47971
W20260808 13:13:28.916451 15241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:28.916782 15241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:28.916924 15241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:28.924275 15241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:28.924401 15241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:28.924563 15241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:28.934638 15241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:41413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=43585
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:28.935971 15241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:28.937306 15241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:28.950197 15246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.950203 15247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:28.950436 15250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:28.951285 15241 server_base.cc:1034] running on GCE node
I20260808 13:13:28.951828 15241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:28.952664 15241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:28.953853 15241 hybrid_clock.cc:648] HybridClock initialized: now 1786194808953801 us; error 42 us; skew 500 ppm
I20260808 13:13:28.954120 15241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:28 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:28.957161 15241 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:28.958648 15241 webserver.cc:460] Webserver started at http://127.31.59.131:43585/ using document root <none> and password file <none>
I20260808 13:13:28.959300 15241 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:28.959427 15241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:28.964653 15241 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 13:13:28.967427 15258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.968676 15241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:13:28.968828 15241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "cf7dfd82989242e69fae1b05a9db04ee"
format_stamp: "Formatted at 2026-08-08 13:13:26 on dist-test-slave-49h3"
server_key: "52731499a70582b1ec18f6b6b2e012b5"
server_key_iv: "5fd5dcba462d40d63241b49ee7e78be1"
server_key_version: "encryptionkey@0"
I20260808 13:13:28.969851 15241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:28.990206 15241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:28.994468 15241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:28.994732 15241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:28.995697 15241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:28.997015 15241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:28.997112 15241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:28.997232 15241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:28.997315 15241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:29.027850 15241 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:41413
I20260808 13:13:29.027935 15376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:41413 every 8 connection(s)
I20260808 13:13:29.029352 15241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:29.038012 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15241
Aug 08 13:13:29 dist-test-slave-49h3 krb5kdc[14266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194808, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:29.050540 15098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.031954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:52859 (local address 127.31.59.190:40209)
0808 13:13:29.032134 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.032139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.033285 (+ 1146us) server_negotiation.cc:407] Connection header received
0808 13:13:29.035171 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.035177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.035266 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.035355 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:29.036818 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.037622 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.039364 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.039853 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.039980 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.044107 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:29.044112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:29.044116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:29.044174 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:29.047012 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.047926 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.047931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.047934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.048129 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.048589 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.048594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.048597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.049007 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:29.049158 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.049900 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.050190 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:13:29.051600 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.032422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:52859)
0808 13:13:29.033117 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.033155 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.034898 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.035604 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.035613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.036110 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:29.036648 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.036665 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.037789 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.037793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.039198 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.039214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.040068 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.040075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.040391 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.041080 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:29.041121 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:29.043893 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:29.047182 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:29.047192 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:29.047213 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:29.047830 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:29.048243 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:29.048248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:29.048251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:29.048472 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:29.049133 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:29.049139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:29.049521 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.049804 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.050341 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260808 13:13:29.053068 15377 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40209
I20260808 13:13:29.053395 15377 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:29.053772 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.038876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.31.59.190:40209)
0808 13:13:29.039364 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.039369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.039711 (+ 342us) server_negotiation.cc:407] Connection header received
0808 13:13:29.039845 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.039850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.039945 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.040038 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:29.042059 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.042771 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.044517 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.045137 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.045278 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.047984 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:29.047991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:29.047995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:29.048031 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:29.051066 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.051875 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.051880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.051884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.051998 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.052385 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.052391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.052394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.052650 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:29.052899 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.053203 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.053393 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260808 13:13:29.054275 15377 heartbeater.cc:511] Master 127.31.59.190:40209 requested a full tablet report, sending...
I20260808 13:13:29.055859 14897 ts_manager.cc:194] Registered new tserver with Master: cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413)
I20260808 13:13:29.056712 14897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:52859
I20260808 13:13:29.068216 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:13:29.074690 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.069060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.31.59.190:40209)
0808 13:13:29.069259 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.069266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.069732 (+ 466us) server_negotiation.cc:407] Connection header received
0808 13:13:29.069905 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.069908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.070051 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.070173 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:13:29.070540 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.071219 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.072598 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.073025 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.073110 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.073646 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:13:29.074139 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:13:29.074262 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.074353 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.074424 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:13:29.075580 14897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:13:29.083097 14897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:13:29.117916 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.095456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.31.59.130:40803)
0808 13:13:29.107726 (+ 12270us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.107736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.107784 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:13:29.107912 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.107921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.108197 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.108426 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:29.109807 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.111545 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.114475 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.116319 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.116460 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.116658 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.117368 (+ 710us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.117572 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12083,"thread_start_us":14048,"threads_started":1}
I20260808 13:13:29.117961 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.095147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:40803 (local address 127.0.0.1:41242)
0808 13:13:29.107217 (+ 12070us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.107240 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.107400 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.108742 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.108763 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.108801 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:29.109338 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.109357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.111704 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.111711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.114306 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.114320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.116830 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.116837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.117087 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.117146 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.117281 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.117523 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11924,"mutex_wait_us":45,"thread_start_us":17116,"threads_started":1}
I20260808 13:13:29.120620 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.096346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.31.59.129:44131)
0808 13:13:29.109484 (+ 13138us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.109494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.112864 (+ 3370us) server_negotiation.cc:407] Connection header received
0808 13:13:29.112968 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.112975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.113190 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.113399 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:29.114178 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.115433 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.117836 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.119305 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.119439 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.119674 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.120164 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.120331 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12938,"thread_start_us":13143,"threads_started":1}
I20260808 13:13:29.120648 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.096067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50620)
0808 13:13:29.112576 (+ 16509us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.112596 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.112732 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.113399 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.113403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.113429 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:29.113925 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.113938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.115559 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.115564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.117640 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.117655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.119830 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.119834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.119984 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.120008 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.120077 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.120208 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16359,"mutex_wait_us":27,"thread_start_us":10877,"threads_started":1}
I20260808 13:13:29.120922 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.094692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:41413 (local address 127.0.0.1:33988)
0808 13:13:29.095496 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.095534 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.095687 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.112832 (+ 17145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.112837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.112860 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:29.113722 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.113741 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.115650 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.115655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.117965 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.117980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.119873 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.119879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.120065 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.120097 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.120181 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.120322 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":73,"thread_start_us":662,"threads_started":1}
I20260808 13:13:29.121848 15388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.094771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.31.59.131:41413)
0808 13:13:29.112201 (+ 17430us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.112209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.112246 (+ 37us) server_negotiation.cc:407] Connection header received
0808 13:13:29.112348 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.112354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.112599 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.113688 (+ 1089us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:29.113989 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.115466 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.118395 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.119354 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.119482 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.119701 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.121345 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.121531 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17263,"thread_start_us":17091,"threads_started":1}
I20260808 13:13:29.122493 15168 tablet_service.cc:1467] Processing CreateTablet for tablet 13552739db284a37b18caea4ce579376 (DEFAULT_TABLE table=test-table [id=a46b0ce986fa4ab2af1a44cc5f5026e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:29.123986 15168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13552739db284a37b18caea4ce579376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:29.124301 15026 tablet_service.cc:1467] Processing CreateTablet for tablet 13552739db284a37b18caea4ce579376 (DEFAULT_TABLE table=test-table [id=a46b0ce986fa4ab2af1a44cc5f5026e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:29.125178 15308 tablet_service.cc:1467] Processing CreateTablet for tablet 13552739db284a37b18caea4ce579376 (DEFAULT_TABLE table=test-table [id=a46b0ce986fa4ab2af1a44cc5f5026e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:29.125906 15026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13552739db284a37b18caea4ce579376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:29.126546 15308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13552739db284a37b18caea4ce579376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:29.136101 15394 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Bootstrap starting.
I20260808 13:13:29.138749 15096 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:29.139786 15394 tablet_bootstrap.cc:654] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:29.141104 15394 log.cc:824] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:29.141217 15395 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Bootstrap starting.
I20260808 13:13:29.145318 15395 tablet_bootstrap.cc:654] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:29.145484 15397 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Bootstrap starting.
I20260808 13:13:29.147073 15395 log.cc:824] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:29.148916 15397 tablet_bootstrap.cc:654] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:29.150264 15397 log.cc:824] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:29.150714 15395 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: No bootstrap required, opened a new log
I20260808 13:13:29.151015 15395 ts_tablet_manager.cc:1397] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 13:13:29.152160 15394 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: No bootstrap required, opened a new log
I20260808 13:13:29.152449 15394 ts_tablet_manager.cc:1397] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260808 13:13:29.152777 15397 tablet_bootstrap.cc:492] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: No bootstrap required, opened a new log
I20260808 13:13:29.153003 15397 ts_tablet_manager.cc:1397] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 13:13:29.160399 15397 raft_consensus.cc:348] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.160771 15397 raft_consensus.cc:374] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:29.160872 15397 raft_consensus.cc:729] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7dfd82989242e69fae1b05a9db04ee, State: Initialized, Role: FOLLOWER
I20260808 13:13:29.161484 15397 consensus_queue.cc:260] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.161527 15395 raft_consensus.cc:348] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.161953 15395 raft_consensus.cc:374] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:29.162160 15395 raft_consensus.cc:729] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f2df7ce6f943a0b921a1ebb512f380, State: Initialized, Role: FOLLOWER
I20260808 13:13:29.162727 15377 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:29.162694 15394 raft_consensus.cc:348] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.163126 15394 raft_consensus.cc:374] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:29.163225 15394 raft_consensus.cc:729] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e77ad1d2154f2fbab93cd74c6a0078, State: Initialized, Role: FOLLOWER
I20260808 13:13:29.163264 15397 ts_tablet_manager.cc:1428] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20260808 13:13:29.163797 15394 consensus_queue.cc:260] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.164368 15395 consensus_queue.cc:260] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.164978 15238 heartbeater.cc:503] Master 127.31.59.190:40209 was elected leader, sending a full tablet report...
I20260808 13:13:29.165606 15394 ts_tablet_manager.cc:1428] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260808 13:13:29.168361 15395 ts_tablet_manager.cc:1428] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260808 13:13:29.201076 15401 raft_consensus.cc:482] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:13:29.201375 15401 raft_consensus.cc:504] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.203106 15401 leader_election.cc:288] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413), d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:29.210309 15388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.203481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:43877 (local address 127.31.59.131:41413)
0808 13:13:29.203675 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.203680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.204317 (+ 637us) server_negotiation.cc:407] Connection header received
0808 13:13:29.204497 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.204503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.204592 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.204699 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:29.205424 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.206383 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.208376 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.209314 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.209438 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.209549 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.209901 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.210044 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:13:29.210317 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.203471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:41413 (local address 127.31.59.130:43877)
0808 13:13:29.204180 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.204199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.204356 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.204848 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.204853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.204883 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:29.205261 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.205274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.206530 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.206537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.208219 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.208232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.209559 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.209565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.209733 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.209768 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.209867 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.209999 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":379,"threads_started":1}
I20260808 13:13:29.210665 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.204317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.31.59.130:57263)
0808 13:13:29.204956 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.204971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:29.205082 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:29.205408 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:29.205412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:29.205435 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:29.205816 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.205828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.207127 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.207131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.208632 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:29.208647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.209909 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.209915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:29.210093 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.210130 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:13:29.210220 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 13:13:29.210347 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":279,"threads_started":1}
I20260808 13:13:29.211002 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.204636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:57263 (local address 127.31.59.129:44131)
0808 13:13:29.204859 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.204865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.205054 (+ 189us) server_negotiation.cc:407] Connection header received
0808 13:13:29.205204 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.205210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.205294 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.205377 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:29.205950 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.206989 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.208800 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.209686 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.209848 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.209917 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.210261 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.210708 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:13:29.211275 15330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13552739db284a37b18caea4ce579376" candidate_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7dfd82989242e69fae1b05a9db04ee" is_pre_election: true
I20260808 13:13:29.211843 15330 raft_consensus.cc:2454] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11e77ad1d2154f2fbab93cd74c6a0078 in term 0.
I20260808 13:13:29.211894 15049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13552739db284a37b18caea4ce579376" candidate_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" is_pre_election: true
I20260808 13:13:29.212416 15049 raft_consensus.cc:2454] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11e77ad1d2154f2fbab93cd74c6a0078 in term 0.
I20260808 13:13:29.212569 15118 leader_election.cc:302] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11e77ad1d2154f2fbab93cd74c6a0078, cf7dfd82989242e69fae1b05a9db04ee; no voters:
I20260808 13:13:29.212960 15401 raft_consensus.cc:2781] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:13:29.213119 15401 raft_consensus.cc:482] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:13:29.213227 15401 raft_consensus.cc:3037] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:29.215865 15401 raft_consensus.cc:504] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.216424 15401 leader_election.cc:288] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [CANDIDATE]: Term 1 election: Requested vote from peers cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413), d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131)
I20260808 13:13:29.216809 15330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13552739db284a37b18caea4ce579376" candidate_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7dfd82989242e69fae1b05a9db04ee"
I20260808 13:13:29.216899 15049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13552739db284a37b18caea4ce579376" candidate_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
I20260808 13:13:29.217003 15330 raft_consensus.cc:3037] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:29.217088 15049 raft_consensus.cc:3037] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:29.219381 15049 raft_consensus.cc:2454] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11e77ad1d2154f2fbab93cd74c6a0078 in term 1.
I20260808 13:13:29.219384 15330 raft_consensus.cc:2454] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11e77ad1d2154f2fbab93cd74c6a0078 in term 1.
I20260808 13:13:29.219714 15118 leader_election.cc:302] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11e77ad1d2154f2fbab93cd74c6a0078, d0f2df7ce6f943a0b921a1ebb512f380; no voters:
I20260808 13:13:29.219976 15401 raft_consensus.cc:2781] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:29.220408 15401 raft_consensus.cc:686] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 LEADER]: Becoming Leader. State: Replica: 11e77ad1d2154f2fbab93cd74c6a0078, State: Running, Role: LEADER
I20260808 13:13:29.220857 15401 consensus_queue.cc:237] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } }
I20260808 13:13:29.225482 14897 catalog_manager.cc:5495] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11e77ad1d2154f2fbab93cd74c6a0078" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 40803 } health_report { overall_health: HEALTHY } } }
I20260808 13:13:29.251013 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.246005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.31.59.130:40803)
0808 13:13:29.246185 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.246189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.246264 (+ 75us) server_negotiation.cc:407] Connection header received
0808 13:13:29.246429 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.246432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.246519 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.246637 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:13:29.246910 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.247675 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.249089 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.249584 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.249675 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.250083 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:13:29.250579 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:13:29.250678 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.250734 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.250799 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:13:29.258868 15330 raft_consensus.cc:1261] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Refusing update from remote peer 11e77ad1d2154f2fbab93cd74c6a0078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:29.258864 15049 raft_consensus.cc:1261] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Refusing update from remote peer 11e77ad1d2154f2fbab93cd74c6a0078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:29.259703 15401 consensus_queue.cc:1035] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7dfd82989242e69fae1b05a9db04ee" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 41413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:29.260010 15406 consensus_queue.cc:1035] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f2df7ce6f943a0b921a1ebb512f380" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 44131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:29.283826 14898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:29.284073 14898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:29.287415 14898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Sending DeleteTablet for 3 replicas of tablet 13552739db284a37b18caea4ce579376
I20260808 13:13:29.288530 15308 tablet_service.cc:1514] Processing DeleteTablet for tablet 13552739db284a37b18caea4ce579376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:29 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33988
I20260808 13:13:29.288566 15026 tablet_service.cc:1514] Processing DeleteTablet for tablet 13552739db284a37b18caea4ce579376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:29 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:50620
I20260808 13:13:29.288684 15168 tablet_service.cc:1514] Processing DeleteTablet for tablet 13552739db284a37b18caea4ce579376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:29 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:41242
I20260808 13:13:29.290015 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14937
I20260808 13:13:29.301331 15420 tablet_replica.cc:330] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: stopping tablet replica
I20260808 13:13:29.302266 15421 tablet_replica.cc:330] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: stopping tablet replica
I20260808 13:13:29.302520 15420 raft_consensus.cc:2229] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:29.302697 15421 raft_consensus.cc:2229] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:29.303179 15420 raft_consensus.cc:2258] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:29.303189 15421 raft_consensus.cc:2258] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:29.303635 15422 tablet_replica.cc:330] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: stopping tablet replica
I20260808 13:13:29.304026 15422 raft_consensus.cc:2229] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:13:29.304720 15422 raft_consensus.cc:2258] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:29.305831 15420 ts_tablet_manager.cc:1905] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:29.306768 15421 ts_tablet_manager.cc:1905] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:29.309226 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.291256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.31.59.129:44131)
0808 13:13:29.291427 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.291433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.291450 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:13:29.291504 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.291509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.291592 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.291698 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:29.293582 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.294792 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.296895 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.297587 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.297714 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.301261 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:29.301277 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:29.301295 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:29.301344 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:29.305941 (+ 4597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.306904 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.306915 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.306918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.307074 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.307510 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.307515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.307518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.308037 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:29.308360 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.308616 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.308870 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:13:29.310153 15091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:29.311141 15422 ts_tablet_manager.cc:1905] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:29.315184 15421 ts_tablet_manager.cc:1918] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:29.315413 15421 log.cc:1196] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/13552739db284a37b18caea4ce579376
I20260808 13:13:29.316042 15421 ts_tablet_manager.cc:1939] T 13552739db284a37b18caea4ce579376 P cf7dfd82989242e69fae1b05a9db04ee: Deleting consensus metadata
I20260808 13:13:29.318193 14880 catalog_manager.cc:4841] TS cf7dfd82989242e69fae1b05a9db04ee (127.31.59.131:41413): tablet 13552739db284a37b18caea4ce579376 (table test-table [id=a46b0ce986fa4ab2af1a44cc5f5026e2]) successfully deleted
I20260808 13:13:29.318778 15422 ts_tablet_manager.cc:1918] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:29.318956 15422 log.cc:1196] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/13552739db284a37b18caea4ce579376
I20260808 13:13:29.319643 15422 ts_tablet_manager.cc:1939] T 13552739db284a37b18caea4ce579376 P 11e77ad1d2154f2fbab93cd74c6a0078: Deleting consensus metadata
I20260808 13:13:29.321357 14882 catalog_manager.cc:4841] TS 11e77ad1d2154f2fbab93cd74c6a0078 (127.31.59.130:40803): tablet 13552739db284a37b18caea4ce579376 (table test-table [id=a46b0ce986fa4ab2af1a44cc5f5026e2]) successfully deleted
I20260808 13:13:29.409781 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14937
W20260808 13:13:29.430158 14880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:44131 (error 108)
I20260808 13:13:29.430415 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 15100
I20260808 13:13:29.432293 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.431772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50632)
0808 13:13:29.431969 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.432091 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 13:13:29.432552 14880 catalog_manager.cc:4584] TS d0f2df7ce6f943a0b921a1ebb512f380 (127.31.59.129:44131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13552739db284a37b18caea4ce579376: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
I20260808 13:13:29.446843 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.431366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.31.59.130:40803)
0808 13:13:29.431570 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.431576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.431598 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:13:29.431753 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.431758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.431869 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.431970 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:29.434279 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.435110 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.436957 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.437402 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.437547 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.440821 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:29.440836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:29.440850 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:29.440889 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:29.443862 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.444682 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.444688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.444691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.444870 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.445288 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.445292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.445294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.445749 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:29.446020 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.446348 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.446519 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:13:29.447569 15233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:29.486320 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.485808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50634)
0808 13:13:29.486015 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.486148 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 13:13:29.487339 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.487003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50648)
0808 13:13:29.487123 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.487205 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 13:13:29.547765 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.547232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50652)
0808 13:13:29.547464 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.547556 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 13:13:29.548833 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.548464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50664)
0808 13:13:29.548617 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.548700 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:13:29.565752 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 15100
I20260808 13:13:29.587534 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 15241
I20260808 13:13:29.600961 15388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.588273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33992 (local address 127.31.59.131:41413)
0808 13:13:29.588442 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:29.588446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:29.588464 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:13:29.588640 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:29.588644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:29.588707 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:29.588810 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:29.590250 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.590976 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.592323 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:29.592724 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:29.592842 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:29.595379 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:29.595394 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:29.595410 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:29.595444 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:29.598174 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.598872 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.598879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.598881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.599028 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:29.599420 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:29.599423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:29.599425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:29.599786 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:29.600081 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:29.600418 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 13:13:29.600649 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 13:13:29.601631 15372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:29.629259 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.628684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50666)
0808 13:13:29.628929 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.629046 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 13:13:29.630916 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.630377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50678)
0808 13:13:29.630665 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.630765 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260808 13:13:29.691469 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 15241
I20260808 13:13:29.710094 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 14860
I20260808 13:13:29.711236 14930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:29.827755 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 14860
I20260808 13:13:29.827805 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:29.827252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:44131 (local address 127.0.0.1:50688)
0808 13:13:29.827460 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:29.827582 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:13:29.850976 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--webserver_interface=127.31.59.190
--webserver_port=38411
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:30.185184 15428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:30.185508 15428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:30.185660 15428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:30.192905 15428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:30.193019 15428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:30.193106 15428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:30.193178 15428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:30.193238 15428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:30.203296 15428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:30.204664 15428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:30.206082 15428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:30.217967 15433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:30.218012 15436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:30.218323 15434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:30.219038 15428 server_base.cc:1034] running on GCE node
I20260808 13:13:30.219805 15428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:30.221212 15428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:30.222420 15428 hybrid_clock.cc:648] HybridClock initialized: now 1786194810222338 us; error 65 us; skew 500 ppm
I20260808 13:13:30.222761 15428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:30 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194810, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:30.225721 15428 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:30.227192 15428 webserver.cc:460] Webserver started at http://127.31.59.190:38411/ using document root <none> and password file <none>
I20260808 13:13:30.227849 15428 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:30.227968 15428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:30.233193 15428 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:30.236053 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:30.237339 15428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:13:30.237517 15428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2bd93266089843de990540ab2c7bfb47"
format_stamp: "Formatted at 2026-08-08 13:13:24 on dist-test-slave-49h3"
server_key: "e88d134b2fefed4e63e08c0382233d72"
server_key_iv: "ca86d73145cc12043d43304c50067486"
server_key_version: "encryptionkey@0"
I20260808 13:13:30.238472 15428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:30.251407 15428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:30.255218 15428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:30.255465 15428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:30.273392 15428 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40209
I20260808 13:13:30.273413 15501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40209 every 8 connection(s)
I20260808 13:13:30.274966 15428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:30.277906 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15428
I20260808 13:13:30.278949 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:44131
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=44029
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:13:30.283563 15502 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:13:30.287909 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap starting.
I20260808 13:13:30.309284 15502 log.cc:824] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:30.325615 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:13:30.326152 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47: Bootstrap complete.
I20260808 13:13:30.335237 15502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:30.336122 15502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Initialized, Role: FOLLOWER
I20260808 13:13:30.336788 15502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:30.337005 15502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:30.337114 15502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:30.337302 15502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:13:30.341065 15502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:30.341586 15502 leader_election.cc:302] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bd93266089843de990540ab2c7bfb47; no voters:
I20260808 13:13:30.342236 15502 leader_election.cc:288] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 13:13:30.342401 15506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:13:30.344352 15502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:30.345779 15506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [term 3 LEADER]: Becoming Leader. State: Replica: 2bd93266089843de990540ab2c7bfb47, State: Running, Role: LEADER
I20260808 13:13:30.346714 15506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } }
I20260808 13:13:30.349206 15508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bd93266089843de990540ab2c7bfb47. Latest consensus state: current_term: 3 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:30.349653 15508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:30.351861 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2bd93266089843de990540ab2c7bfb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bd93266089843de990540ab2c7bfb47" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40209 } } }
I20260808 13:13:30.352119 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bd93266089843de990540ab2c7bfb47 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:30.358106 15515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:30.366354 15515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:30.367836 15515 catalog_manager.cc:1259] Loaded cluster ID: f17ef3f597c3447ca60456d59e7cd03e
I20260808 13:13:30.367944 15515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 13:13:30.370052 15515 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa5bbbc69ad google::LogMessage::Fail() at ??:0
@ 0x7fa5bbbc988a google::LogMessage::SendToLog() at ??:0
@ 0x7fa5bbbc64de google::LogMessage::Flush() at ??:0
@ 0x7fa5bbbc7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa5d15f9512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa5d15f6976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa5d167e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa5d167e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa5be812345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa5be81591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa5be815799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa5be7f0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa5c1aca6db start_thread at ??:0
@ 0x7fa5baa6061f clone at ??:0
W20260808 13:13:30.714798 15504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:30.715080 15504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:30.715178 15504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:30.722539 15504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:30.722661 15504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:30.722784 15504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:30.732580 15504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:44131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=44029
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:30.733867 15504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:30.735194 15504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:30.747577 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:30.747613 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:30.747659 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:30.748756 15504 server_base.cc:1034] running on GCE node
I20260808 13:13:30.749400 15504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:30.750425 15504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:30.751639 15504 hybrid_clock.cc:648] HybridClock initialized: now 1786194810751576 us; error 41 us; skew 500 ppm
I20260808 13:13:30.751950 15504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:30 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194810, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:30.755002 15504 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:30.756479 15504 webserver.cc:460] Webserver started at http://127.31.59.129:44029/ using document root <none> and password file <none>
I20260808 13:13:30.757145 15504 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:30.757268 15504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:30.762591 15504 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:13:30.765439 15555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:30.766810 15504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:13:30.766963 15504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "d0f2df7ce6f943a0b921a1ebb512f380"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "36cc49179e3e70e9a5c5df94dc0b70e3"
server_key_iv: "1da42c6df2be779c266f5aee53790e62"
server_key_version: "encryptionkey@0"
I20260808 13:13:30.767952 15504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:30.785487 15504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:30.789268 15504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:30.789566 15504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:30.790580 15504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:30.792346 15562 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260808 13:13:30.797600 15562 ts_tablet_manager.cc:1852] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 13:13:30.798662 15562 ts_tablet_manager.cc:1905] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:30.806051 15562 ts_tablet_manager.cc:1918] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:30.806259 15562 log.cc:1196] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/13552739db284a37b18caea4ce579376
I20260808 13:13:30.806713 15562 ts_tablet_manager.cc:1939] T 13552739db284a37b18caea4ce579376 P d0f2df7ce6f943a0b921a1ebb512f380: Deleting consensus metadata
I20260808 13:13:30.807107 15504 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 13:13:30.807226 15504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260808 13:13:30.807344 15504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:30.807428 15504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:30.839004 15504 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:44131
I20260808 13:13:30.839071 15674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:44131 every 8 connection(s)
I20260808 13:13:30.840358 15504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:30.844960 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:30.843179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:42643)
0808 13:13:30.843888 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:30.844136 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":270,"threads_started":1}
W20260808 13:13:30.845537 15675 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:40209 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:40209: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
I20260808 13:13:30.846307 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:30.845997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:47029)
0808 13:13:30.846130 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:30.846198 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:13:30.847060 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:30.846774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:36315)
0808 13:13:30.846884 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:30.846952 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 13:13:30.847266 15675 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:13:30.850800 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15504
I20260808 13:13:30.851394 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:40803
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=41969
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:31.188369 15678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:31.188676 15678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:31.188812 15678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:31.196467 15678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:31.196616 15678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:31.196768 15678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:31.206893 15678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=41969
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:31.208320 15678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:31.209748 15678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:31.223008 15684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:31.222966 15685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:31.223031 15687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:31.223878 15678 server_base.cc:1034] running on GCE node
I20260808 13:13:31.224540 15678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:31.225585 15678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:31.226804 15678 hybrid_clock.cc:648] HybridClock initialized: now 1786194811226741 us; error 54 us; skew 500 ppm
I20260808 13:13:31.227123 15678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:31 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194811, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:31.230216 15678 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:31.231698 15678 webserver.cc:460] Webserver started at http://127.31.59.130:41969/ using document root <none> and password file <none>
I20260808 13:13:31.232383 15678 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:31.232506 15678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:31.238040 15678 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:13:31.240927 15694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.242286 15678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:31.242455 15678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "11e77ad1d2154f2fbab93cd74c6a0078"
format_stamp: "Formatted at 2026-08-08 13:13:25 on dist-test-slave-49h3"
server_key: "3da41418b7d671cbb5c8c6ed9498b9ad"
server_key_iv: "a50f041506d8bd60f7c4808d6e9adb20"
server_key_version: "encryptionkey@0"
I20260808 13:13:31.243453 15678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:31.259218 15678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:31.263193 15678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:31.263487 15678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:31.264523 15678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:31.266079 15678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:31.266186 15678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.266304 15678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:31.266376 15678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.298451 15678 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:40803
I20260808 13:13:31.298511 15815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:40803 every 8 connection(s)
I20260808 13:13:31.299929 15678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:31.304697 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.302800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:44859)
0808 13:13:31.303610 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.303835 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":340,"threads_started":1}
W20260808 13:13:31.305375 15816 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:40209 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:40209: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
I20260808 13:13:31.306210 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.305882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:41693)
0808 13:13:31.306014 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.306097 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 13:13:31.307029 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.306697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:43643)
0808 13:13:31.306831 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.306907 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:13:31.307308 15816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:13:31.308604 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15678
I20260808 13:13:31.309190 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:41413
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:40209
--webserver_port=43585
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:31.647092 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:31.647419 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:31.647537 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:31.654940 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:31.655054 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:31.655169 15819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:31.664939 15819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:41413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=43585
--tserver_master_addrs=127.31.59.190:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:31.666411 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:31.667806 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:31.680837 15825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:31.680796 15828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:31.680876 15826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:31.681950 15819 server_base.cc:1034] running on GCE node
I20260808 13:13:31.682503 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:31.683367 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:31.684561 15819 hybrid_clock.cc:648] HybridClock initialized: now 1786194811684509 us; error 43 us; skew 500 ppm
I20260808 13:13:31.684860 15819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:31 dist-test-slave-49h3 krb5kdc[14266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194811, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:31.687978 15819 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:31.689465 15819 webserver.cc:460] Webserver started at http://127.31.59.131:43585/ using document root <none> and password file <none>
I20260808 13:13:31.690114 15819 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:31.690234 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:31.695423 15819 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 13:13:31.698068 15835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.699314 15819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:31.699460 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "cf7dfd82989242e69fae1b05a9db04ee"
format_stamp: "Formatted at 2026-08-08 13:13:26 on dist-test-slave-49h3"
server_key: "52731499a70582b1ec18f6b6b2e012b5"
server_key_iv: "5fd5dcba462d40d63241b49ee7e78be1"
server_key_version: "encryptionkey@0"
I20260808 13:13:31.700420 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:31.719547 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:31.723758 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:31.724014 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:31.725029 15819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:31.726372 15819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:31.726465 15819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.726575 15819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:31.726642 15819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:31.757864 15819 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:41413
I20260808 13:13:31.757936 15953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:41413 every 8 connection(s)
I20260808 13:13:31.759307 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:31.763898 15958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.762087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:46685)
0808 13:13:31.762814 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.763044 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":287,"threads_started":1}
W20260808 13:13:31.764498 15956 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:40209 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:40209: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
I20260808 13:13:31.765231 15958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.764931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:56489)
0808 13:13:31.765061 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.765127 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 13:13:31.766016 15958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.765743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:49051)
0808 13:13:31.765844 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.765914 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 13:13:31.766249 15956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:13:31.768608 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 15819
I20260808 13:13:31.848605 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:31.847804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:38171)
0808 13:13:31.848267 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:31.848358 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":208,"threads_started":1}
I20260808 13:13:32.308578 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:32.307806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:48697)
0808 13:13:32.308242 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:32.308356 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":208,"threads_started":1}
I20260808 13:13:32.767663 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:32.766799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:53355)
0808 13:13:32.767313 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:32.767429 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":244,"threads_started":1}
I20260808 13:13:32.850318 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:32.849609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:51805)
0808 13:13:32.850064 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:32.850161 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":240,"threads_started":1}
I20260808 13:13:33.310163 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:33.309378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:37457)
0808 13:13:33.309890 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:33.309976 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":275,"threads_started":1}
I20260808 13:13:33.769500 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:33.768688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:34749)
0808 13:13:33.769176 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:33.769278 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":230,"threads_started":1}
I20260808 13:13:33.851941 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:33.851214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:54237)
0808 13:13:33.851677 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:33.851765 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":230,"threads_started":1}
I20260808 13:13:34.311734 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:34.310999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:56701)
0808 13:13:34.311469 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:34.311554 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260808 13:13:34.771160 15968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:34.770430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:53173)
0808 13:13:34.770894 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:34.770983 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260808 13:13:34.853480 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:34.852773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:40659)
0808 13:13:34.853209 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:34.853293 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":218,"threads_started":1}
I20260808 13:13:35.313511 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:35.312643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:38363)
0808 13:13:35.313163 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:35.313259 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
I20260808 13:13:35.772764 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:35.772046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:56087)
0808 13:13:35.772484 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:35.772563 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":213,"threads_started":1}
I20260808 13:13:35.855079 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:35.854338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:57085)
0808 13:13:35.854819 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:35.854905 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":225,"threads_started":1}
I20260808 13:13:36.315361 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:36.314479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:54777)
0808 13:13:36.315065 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:36.315178 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":295,"threads_started":1}
I20260808 13:13:36.774441 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:36.773670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:56619)
0808 13:13:36.774165 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:36.774262 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":260,"threads_started":1}
I20260808 13:13:36.856705 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:36.855956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:35713)
0808 13:13:36.856447 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:36.856538 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":272,"threads_started":1}
I20260808 13:13:37.316917 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:37.316187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:41433)
0808 13:13:37.316653 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:37.316749 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":242,"threads_started":1}
I20260808 13:13:37.776026 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:37.775317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:50719)
0808 13:13:37.775763 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:37.775857 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":228,"threads_started":1}
I20260808 13:13:37.858418 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:37.857573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:46143)
0808 13:13:37.858108 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:37.858197 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":251,"threads_started":1}
I20260808 13:13:38.318638 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:38.317838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:56159)
0808 13:13:38.318334 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:38.318420 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":208,"threads_started":1}
I20260808 13:13:38.777750 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:38.776902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:51951)
0808 13:13:38.777389 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:38.777524 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":207,"threads_started":1}
I20260808 13:13:38.859988 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:38.859270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:40777)
0808 13:13:38.859718 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:38.859818 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":241,"threads_started":1}
I20260808 13:13:39.320257 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:39.319527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:33949)
0808 13:13:39.319991 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:39.320074 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":230,"threads_started":1}
I20260808 13:13:39.779341 15985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:39.778634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:49143)
0808 13:13:39.779085 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:39.779170 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":225,"threads_started":1}
I20260808 13:13:39.861598 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:39.860825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:47693)
0808 13:13:39.861291 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:39.861390 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":213,"threads_started":1}
I20260808 13:13:40.322234 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:40.321168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:36803)
0808 13:13:40.321908 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:40.322025 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":386,"threads_started":1}
I20260808 13:13:40.781067 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:40.780281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:56815)
0808 13:13:40.780793 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:40.780881 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260808 13:13:40.863405 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:40.862615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:47061)
0808 13:13:40.863134 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:40.863229 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":251,"threads_started":1}
I20260808 13:13:41.323854 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:41.323153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:50573)
0808 13:13:41.323602 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:41.323688 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":249,"threads_started":1}
I20260808 13:13:41.782823 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:41.782078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:35901)
0808 13:13:41.782547 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:41.782634 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":258,"threads_started":1}
I20260808 13:13:41.865300 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:41.864402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:36019)
0808 13:13:41.865000 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:41.865089 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":341,"threads_started":1}
I20260808 13:13:42.325604 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:42.324795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:50209)
0808 13:13:42.325281 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:42.325378 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":257,"threads_started":1}
I20260808 13:13:42.784387 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:42.783669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:47277)
0808 13:13:42.784110 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:42.784199 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":222,"threads_started":1}
I20260808 13:13:42.866999 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:42.866266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:56109)
0808 13:13:42.866741 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:42.866832 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":243,"threads_started":1}
I20260808 13:13:43.327229 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:43.326519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:48657)
0808 13:13:43.326982 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:43.327066 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":244,"threads_started":1}
I20260808 13:13:43.785969 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:43.785224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:39983)
0808 13:13:43.785706 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:43.785806 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":249,"threads_started":1}
I20260808 13:13:43.868747 16000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:43.867935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:50473)
0808 13:13:43.868427 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:43.868539 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":233,"threads_started":1}
I20260808 13:13:44.328905 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:44.328125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:43591)
0808 13:13:44.328614 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:44.328710 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":253,"threads_started":1}
I20260808 13:13:44.787602 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:44.786848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:59301)
0808 13:13:44.787321 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:44.787412 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":221,"threads_started":1}
I20260808 13:13:44.870383 16003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:44.869635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:39453)
0808 13:13:44.870101 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:44.870183 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":244,"threads_started":1}
I20260808 13:13:45.330655 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:45.329843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:33369)
0808 13:13:45.330311 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:45.330422 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":231,"threads_started":1}
I20260808 13:13:45.789192 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:45.788423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:39889)
0808 13:13:45.788916 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:45.789007 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":254,"threads_started":1}
I20260808 13:13:45.872336 16006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:45.871549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:43643)
0808 13:13:45.872042 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:45.872138 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":241,"threads_started":1}
I20260808 13:13:46.332603 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:46.331863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.130:35913)
0808 13:13:46.332303 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:46.332400 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":221,"threads_started":1}
I20260808 13:13:46.772039 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 15504
I20260808 13:13:46.789513 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:46.773087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.31.59.129:44131)
0808 13:13:46.773663 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:46.773677 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:46.773717 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:13:46.775679 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:46.775690 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:46.776132 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:46.776431 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:46.777859 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:46.779112 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:46.780362 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:46.781016 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:46.781152 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:46.783703 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:46.783710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:46.783726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:46.783767 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:46.786632 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:46.787231 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:46.787237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:46.787252 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:46.787411 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:46.787696 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:46.787699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:46.787701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:46.788207 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:46.788465 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:46.788756 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 13:13:46.789148 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":254,"threads_started":1}
I20260808 13:13:46.791010 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:46.790236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.131:58485)
0808 13:13:46.790738 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:46.790833 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":234,"threads_started":1}
I20260808 13:13:46.791035 15669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:46.876374 16011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:46.875194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40209 (local address 127.31.59.129:54913)
0808 13:13:46.875788 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:46.876039 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:40209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":260,"threads_started":1}
I20260808 13:13:46.876380 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 15504
I20260808 13:13:46.897951 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 15678
I20260808 13:13:46.915081 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:46.898867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.31.59.130:40803)
0808 13:13:46.899390 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:46.899406 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:46.899442 (+ 36us) server_negotiation.cc:407] Connection header received
0808 13:13:46.900999 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:46.901008 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:46.901431 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:46.901745 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:46.903050 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:46.904284 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:46.905632 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:46.906324 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:46.906502 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:46.909065 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:46.909072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:46.909087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:46.909124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:46.912006 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:46.912815 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:46.912822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:46.912829 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:46.913007 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:46.913306 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:46.913309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:46.913311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:46.913879 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:46.914106 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:46.914416 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 13:13:46.914791 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":230,"threads_started":1}
I20260808 13:13:46.916409 15808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:47.009827 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 15678
I20260808 13:13:47.028481 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 15819
I20260808 13:13:47.046142 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:47.029536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.31.59.131:41413)
0808 13:13:47.030145 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:47.030159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:47.030201 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:13:47.031874 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:47.031884 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:47.032310 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:47.032633 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:47.034171 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:47.035372 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:47.036616 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:47.037276 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:47.037469 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:47.040107 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:47.040115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:47.040130 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:47.040168 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:47.043077 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:47.043742 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:47.043749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:47.043755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:47.043931 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:47.044218 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:47.044221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:47.044222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:47.044746 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:47.044987 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:47.045387 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 13:13:47.045821 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":273,"threads_started":1}
I20260808 13:13:47.047459 15949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:47.130519 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 15819
2026-08-08T13:13:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24333 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:13:47 dist-test-slave-49h3 krb5kdc[16018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:13:47 dist-test-slave-49h3 krb5kdc[16018](info): set up 2 sockets
Aug 08 13:13:47 dist-test-slave-49h3 krb5kdc[16018](info): commencing operation
krb5kdc: starting...
W20260808 13:13:48.217190 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:13:48 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:13:48.520051 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:36101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:48.856827 16032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:48.857142 16032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:48.857290 16032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:48.864495 16032 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:13:48.864598 16032 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:13:48.864640 16032 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:13:48.864713 16032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:13:48.864787 16032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:48.864884 16032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:48.864941 16032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:13:48.865016 16032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:13:48.875237 16032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:36101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:48.876590 16032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:48.877959 16032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:48.889981 16038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:48.889894 16041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:48.889937 16039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:48.890693 16032 server_base.cc:1034] running on GCE node
I20260808 13:13:48.891480 16032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:48.892895 16032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:48.894106 16032 hybrid_clock.cc:648] HybridClock initialized: now 1786194828894041 us; error 35 us; skew 500 ppm
I20260808 13:13:48.894465 16032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:48 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:48.901834 16032 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:13:48.903260 16032 webserver.cc:460] Webserver started at http://127.31.59.190:39945/ using document root <none> and password file <none>
I20260808 13:13:48.903931 16032 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:48.904027 16032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:48.904417 16032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:48.907462 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "150b8fe8a4704874bc8a4903bd44ec33"
format_stamp: "Formatted at 2026-08-08 13:13:48 on dist-test-slave-49h3"
server_key: "2ac3ea081b22bf89622396be2733c752"
server_key_iv: "51878987c84655377b20baba1a327118"
server_key_version: "encryptionkey@0"
I20260808 13:13:48.908245 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "150b8fe8a4704874bc8a4903bd44ec33"
format_stamp: "Formatted at 2026-08-08 13:13:48 on dist-test-slave-49h3"
server_key: "2ac3ea081b22bf89622396be2733c752"
server_key_iv: "51878987c84655377b20baba1a327118"
server_key_version: "encryptionkey@0"
I20260808 13:13:48.913399 16032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:13:48.916807 16049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:48.918103 16032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:48.918291 16032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "150b8fe8a4704874bc8a4903bd44ec33"
format_stamp: "Formatted at 2026-08-08 13:13:48 on dist-test-slave-49h3"
server_key: "2ac3ea081b22bf89622396be2733c752"
server_key_iv: "51878987c84655377b20baba1a327118"
server_key_version: "encryptionkey@0"
I20260808 13:13:48.918491 16032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:48.944535 16032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:48.948901 16032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:48.949203 16032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:48.967195 16032 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:36101
I20260808 13:13:48.967216 16101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:36101 every 8 connection(s)
I20260808 13:13:48.968672 16032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:13:48.972570 16102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:48.978345 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16032
I20260808 13:13:48.978559 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:13:48.979242 31982 external_mini_cluster.cc:1428] Setting key 00e9c022310895a34809bc940d19ed78
I20260808 13:13:48.983493 16102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap starting.
I20260808 13:13:48.987192 16102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:48.988242 16102 log.cc:824] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:48.991358 16102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: No bootstrap required, opened a new log
Aug 08 13:13:48 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:48.999862 16102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:13:49.000247 16102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:49.000344 16102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Initialized, Role: FOLLOWER
I20260808 13:13:49.000937 16102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:13:49.001185 16102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:13:49.001353 16102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:13:49.001609 16102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:49.002750 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:48.981512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.31.59.190:36101)
0808 13:13:48.982394 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:48.982407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:48.982451 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:13:48.984226 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:48.984249 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:48.984633 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:48.984962 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:48.986781 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:48.988131 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:48.989874 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:48.990556 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:48.990689 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:48.995362 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:48.995370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:48.995385 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:48.995421 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:48.998583 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:48.999289 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:48.999296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:48.999303 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:48.999470 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:48.999843 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:48.999851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:48.999853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:49.000433 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:49.000740 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:49.001180 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 13:13:49.001659 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":394,"threads_started":1}
I20260808 13:13:49.004590 16102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:13:49.005069 16102 leader_election.cc:302] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150b8fe8a4704874bc8a4903bd44ec33; no voters:
I20260808 13:13:49.005770 16102 leader_election.cc:288] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:13:49.005877 16107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:49.006556 16107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 LEADER]: Becoming Leader. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Running, Role: LEADER
I20260808 13:13:49.007182 16107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:13:49.007895 16102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:13:49.010219 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8fe8a4704874bc8a4903bd44ec33. Latest consensus state: current_term: 1 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:13:49.010210 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:13:49.010773 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:49.010797 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:13:49.018754 16113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:13:49.024271 16113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:13:49.032699 16113 catalog_manager.cc:1347] Generated new cluster ID: d87a62e679bd4dd9b072d35c7f934483
I20260808 13:13:49.032825 16113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:13:49.044124 16113 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:13:49.045511 16113 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:13:49.054754 16113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Generated new TSK 0
I20260808 13:13:49.055706 16113 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:13:49.146685 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:49.487046 16129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:49.487322 16129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:49.487425 16129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:49.494659 16129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:49.494769 16129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:49.494890 16129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:13:49.504915 16129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:49.506210 16129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:49.507503 16129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:49.520308 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:49.520274 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:49.520434 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:49.521247 16129 server_base.cc:1034] running on GCE node
I20260808 13:13:49.521924 16129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:49.522833 16129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:49.524044 16129 hybrid_clock.cc:648] HybridClock initialized: now 1786194829523981 us; error 53 us; skew 500 ppm
I20260808 13:13:49.524355 16129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:49 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:49.527544 16129 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:13:49.528975 16129 webserver.cc:460] Webserver started at http://127.31.59.129:44501/ using document root <none> and password file <none>
I20260808 13:13:49.529695 16129 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:49.529824 16129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:49.530122 16129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:49.533027 16129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
format_stamp: "Formatted at 2026-08-08 13:13:49 on dist-test-slave-49h3"
server_key: "e286a5944fc8e1f50cc4da5ed68daca5"
server_key_iv: "c6ffeb5bda6db3564f6f5ef721015f61"
server_key_version: "encryptionkey@0"
I20260808 13:13:49.533842 16129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
format_stamp: "Formatted at 2026-08-08 13:13:49 on dist-test-slave-49h3"
server_key: "e286a5944fc8e1f50cc4da5ed68daca5"
server_key_iv: "c6ffeb5bda6db3564f6f5ef721015f61"
server_key_version: "encryptionkey@0"
I20260808 13:13:49.539069 16129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:13:49.542412 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:49.543641 16129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:13:49.543804 16129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
format_stamp: "Formatted at 2026-08-08 13:13:49 on dist-test-slave-49h3"
server_key: "e286a5944fc8e1f50cc4da5ed68daca5"
server_key_iv: "c6ffeb5bda6db3564f6f5ef721015f61"
server_key_version: "encryptionkey@0"
I20260808 13:13:49.543979 16129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:49.571060 16129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:49.575376 16129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:49.575661 16129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:49.576639 16129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:49.577997 16129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:49.578076 16129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:49.578198 16129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:49.578253 16129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:49.609753 16129 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45387
I20260808 13:13:49.609823 16269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45387 every 8 connection(s)
I20260808 13:13:49.611068 16129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:13:49.615943 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16129
I20260808 13:13:49.616154 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:13:49.616793 31982 external_mini_cluster.cc:1428] Setting key c8ac8fbe65e2cbdf26eef074fca7868f
Aug 08 13:13:49 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:49.631474 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:49.613674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55633 (local address 127.31.59.190:36101)
0808 13:13:49.614095 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:49.614099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:49.615139 (+ 1040us) server_negotiation.cc:407] Connection header received
0808 13:13:49.617100 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:49.617106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:49.617221 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:49.617350 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:49.619058 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:49.620107 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:49.621672 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:49.622207 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:49.622320 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:49.625613 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:49.625618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:49.625622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:49.625661 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:49.628624 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:49.629298 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:49.629303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:49.629307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:49.629483 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:49.629867 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:49.629871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:49.629874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:49.630185 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:49.630338 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:49.630942 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 13:13:49.631153 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":201,"threads_started":1}
I20260808 13:13:49.632262 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:49.614235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:55633)
0808 13:13:49.614967 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:49.615013 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:49.616812 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:49.617617 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:49.617629 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:49.618189 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:49.618892 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:49.618915 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:49.620253 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:49.620258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:49.621540 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:49.621557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:49.622399 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:49.622404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:49.622598 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:49.623255 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:49.623277 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:49.625378 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:49.628756 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:49.628763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:49.628774 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:49.629177 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:49.629568 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:49.629571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:49.629573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:49.629765 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:49.630283 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:49.630288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:49.630581 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 13:13:49.630804 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 13:13:49.631178 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":290,"threads_started":1}
I20260808 13:13:49.633813 16270 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:13:49.634117 16270 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:49.634835 16270 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:13:49.637099 16066 ts_manager.cc:194] Registered new tserver with Master: cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387)
I20260808 13:13:49.639066 16066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:55633
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:13:49.682762 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:50.027837 16276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:50.028142 16276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:50.028287 16276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:50.035626 16276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:50.035763 16276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:50.035921 16276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:13:50.046025 16276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:50.047360 16276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:50.048722 16276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:50.061632 16281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:50.061666 16282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:50.061695 16284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:50.062669 16276 server_base.cc:1034] running on GCE node
I20260808 13:13:50.063239 16276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:50.064131 16276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:50.065346 16276 hybrid_clock.cc:648] HybridClock initialized: now 1786194830065289 us; error 45 us; skew 500 ppm
I20260808 13:13:50.065689 16276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:50 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194830, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:50.068753 16276 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:13:50.070236 16276 webserver.cc:460] Webserver started at http://127.31.59.130:46585/ using document root <none> and password file <none>
I20260808 13:13:50.070887 16276 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:50.071018 16276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:50.071308 16276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:50.074230 16276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "66a0502ef12b36fa4c46099bbe197f2f"
server_key_iv: "638d3af540400641b53af54023fe4a28"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.074962 16276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "66a0502ef12b36fa4c46099bbe197f2f"
server_key_iv: "638d3af540400641b53af54023fe4a28"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.079888 16276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:13:50.083299 16293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.084613 16276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:13:50.084769 16276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "66a0502ef12b36fa4c46099bbe197f2f"
server_key_iv: "638d3af540400641b53af54023fe4a28"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.084964 16276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:50.107035 16276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:50.111370 16276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:50.111665 16276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:50.112713 16276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:50.114167 16276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:50.114259 16276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.114387 16276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:50.114441 16276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.146456 16276 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34003
I20260808 13:13:50.146528 16413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34003 every 8 connection(s)
I20260808 13:13:50.147718 16276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:13:50.151989 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16276
I20260808 13:13:50.152190 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:13:50.152629 31982 external_mini_cluster.cc:1428] Setting key 4c8a7a04db011cd0666c23b194335505
Aug 08 13:13:50 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194830, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:50.168992 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.150302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:45605 (local address 127.31.59.190:36101)
0808 13:13:50.150777 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.150782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.151589 (+ 807us) server_negotiation.cc:407] Connection header received
0808 13:13:50.153721 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.153727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.153813 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.153924 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:50.155784 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.156545 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.158610 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.159061 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.159165 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.162997 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:50.163003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:50.163008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:50.163047 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:50.166098 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.166807 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.166812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.166815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.166960 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.167368 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.167372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.167375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.167688 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:50.167824 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.168547 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.168722 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
I20260808 13:13:50.170506 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.150679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:45605)
0808 13:13:50.151429 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.151467 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.153302 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.154126 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.154133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.154780 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:50.155567 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.155595 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.156694 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.156700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.158459 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.158475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.159291 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.159298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.159574 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.160265 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:50.160293 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:50.162745 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:50.166225 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:50.166231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:50.166250 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:50.166683 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:50.167048 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:50.167051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:50.167053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:50.167243 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:50.167788 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:50.167794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:50.168124 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.168459 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.169004 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":308,"threads_started":1}
I20260808 13:13:50.171962 16414 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:13:50.172288 16414 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:50.173090 16414 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:13:50.174654 16066 ts_manager.cc:194] Registered new tserver with Master: b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
I20260808 13:13:50.175464 16066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:45605
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:13:50.220351 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:13:50.551815 16420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:13:50.552101 16420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:13:50.552251 16420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:13:50.559653 16420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:13:50.559788 16420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:13:50.559942 16420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:13:50.569715 16420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:13:50.571022 16420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:13:50.572366 16420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:13:50.585016 16425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:50.585047 16426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:13:50.585083 16428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:13:50.585700 16420 server_base.cc:1034] running on GCE node
I20260808 13:13:50.586290 16420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:13:50.587230 16420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:13:50.588436 16420 hybrid_clock.cc:648] HybridClock initialized: now 1786194830588390 us; error 30 us; skew 500 ppm
I20260808 13:13:50.588714 16420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:13:50 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194830, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:13:50.591838 16420 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:13:50.593194 16420 webserver.cc:460] Webserver started at http://127.31.59.131:34285/ using document root <none> and password file <none>
I20260808 13:13:50.593940 16420 fs_manager.cc:359] Metadata directory not provided
I20260808 13:13:50.594046 16420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:13:50.594300 16420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:13:50.597240 16420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "8463024245f441629bee3e2615d8eb2d"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "0bb135859782ceb024f53550eba09fb6"
server_key_iv: "15fcb56d979ad9273a35821cdcdf100c"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.598001 16420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "8463024245f441629bee3e2615d8eb2d"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "0bb135859782ceb024f53550eba09fb6"
server_key_iv: "15fcb56d979ad9273a35821cdcdf100c"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.608901 16420 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.010s sys 0.000s
I20260808 13:13:50.612701 16436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.614149 16420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:13:50.614315 16420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8463024245f441629bee3e2615d8eb2d"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "0bb135859782ceb024f53550eba09fb6"
server_key_iv: "15fcb56d979ad9273a35821cdcdf100c"
server_key_version: "encryptionkey@0"
I20260808 13:13:50.614496 16420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:13:50.629311 16420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:13:50.632925 16420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:13:50.633177 16420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:13:50.634143 16420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:13:50.635445 16420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:13:50.635524 16420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.635648 16420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:13:50.635706 16420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:13:50.642269 16270 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:13:50.666306 16420 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:39335
I20260808 13:13:50.666352 16554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:39335 every 8 connection(s)
I20260808 13:13:50.667604 16420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:13:50.668480 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16420
I20260808 13:13:50.668664 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:13:50.669229 31982 external_mini_cluster.cc:1428] Setting key 219b1fafbda8e49a0edf1f7ac18ab59c
Aug 08 13:13:50 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194830, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:13:50.687777 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.671013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:53247 (local address 127.31.59.190:36101)
0808 13:13:50.671666 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.671671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.672616 (+ 945us) server_negotiation.cc:407] Connection header received
0808 13:13:50.674489 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.674494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.674576 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.674666 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:50.676147 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.676931 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.678495 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.678899 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.679007 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.682266 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:50.682271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:50.682274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:50.682305 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:50.684850 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.685543 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.685548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.685550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.685665 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.686085 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.686089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.686091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.686357 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:50.686501 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.687173 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.687395 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":416,"threads_started":1}
I20260808 13:13:50.688722 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.671601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:53247)
0808 13:13:50.672373 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.672418 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.674242 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.674906 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.674915 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.675426 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:13:50.675984 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.676002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.677084 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.677088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.678362 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.678374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.679123 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.679131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.679363 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.679933 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:13:50.679954 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:13:50.682068 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:13:50.684979 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:50.684988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:50.685003 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:50.685418 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:50.685766 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:13:50.685771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:13:50.685775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:13:50.685961 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:13:50.686487 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:13:50.686495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:13:50.686816 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.687044 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.687558 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":324,"threads_started":1}
I20260808 13:13:50.689944 16555 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:13:50.690219 16555 heartbeater.cc:464] Registering TS with master...
I20260808 13:13:50.690923 16555 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:13:50.692229 16066 ts_manager.cc:194] Registered new tserver with Master: 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335)
I20260808 13:13:50.692989 16066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:53247
I20260808 13:13:50.697273 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:13:50.713768 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.701979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.31.59.190:36101)
0808 13:13:50.702176 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.702183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.702596 (+ 413us) server_negotiation.cc:407] Connection header received
0808 13:13:50.702755 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.702759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.702824 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.702895 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:50.704271 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.705021 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.706404 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.706809 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.706897 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.709097 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:50.709102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:50.709105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:50.709136 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:50.711575 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.712150 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.712153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.712156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.712269 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:50.712527 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:50.712530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:50.712531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:50.712837 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:50.713017 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.713273 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.713438 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:13:50.717819 16066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:13:50.724933 16066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:13:50.748131 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.737109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59690)
0808 13:13:50.737979 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.738041 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.738271 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.739554 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.739561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.739592 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:50.740036 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.740048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.742427 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.742433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.744925 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.744942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.747061 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.747069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.747321 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.747386 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.747501 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.747716 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":900,"threads_started":1}
I20260808 13:13:50.749908 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.737523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.31.59.129:45387)
0808 13:13:50.738907 (+ 1384us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.738915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.738946 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:13:50.739042 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.739050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.739319 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.740241 (+ 922us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:50.740489 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.742257 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.745152 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.746817 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.748985 (+ 2168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.749279 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.749356 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.749556 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":1070,"threads_started":1}
I20260808 13:13:50.754668 16195 tablet_service.cc:1467] Processing CreateTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 (DEFAULT_TABLE table=test-table [id=e36ef8070a524e4090772d2b4e79b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:50.756287 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 109da5d315a94e2d8783f9f39a2bcda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:50.757617 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.738036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.0.0.1:43734)
0808 13:13:50.739190 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.739212 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.739320 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.751079 (+ 11759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.751084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.751110 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:50.751421 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.751431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.753252 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.753258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.755019 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.755033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.756794 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.756802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.756969 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.757004 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.757098 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.757241 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":1390,"threads_started":1}
I20260808 13:13:50.757673 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.738574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.31.59.130:34003)
0808 13:13:50.750406 (+ 11832us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.750416 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.750449 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:13:50.750579 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.750588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.750879 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.751074 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:50.751582 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.753098 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.755177 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.756583 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.756737 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.756934 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.757149 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.757322 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11637,"thread_start_us":12091,"threads_started":1}
I20260808 13:13:50.758548 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.738827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39335 (local address 127.0.0.1:38964)
0808 13:13:50.748424 (+ 9597us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.748445 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.748552 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.752742 (+ 4190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.752747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.752776 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:50.752978 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.752990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.755035 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.755039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.756225 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.756237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.758034 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.758040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.758204 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.758227 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.758279 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.758371 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9491,"thread_start_us":11681,"threads_started":1}
I20260808 13:13:50.759709 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.739193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.31.59.131:39335)
0808 13:13:50.752167 (+ 12974us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.752175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.752207 (+ 32us) server_negotiation.cc:407] Connection header received
0808 13:13:50.752310 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.752318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.752521 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.752709 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:50.753326 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.754871 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.756384 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.757889 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.758947 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.759167 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.759235 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.759387 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12805,"thread_start_us":12645,"threads_started":1}
I20260808 13:13:50.761157 16344 tablet_service.cc:1467] Processing CreateTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 (DEFAULT_TABLE table=test-table [id=e36ef8070a524e4090772d2b4e79b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:50.762656 16344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 109da5d315a94e2d8783f9f39a2bcda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:50.763406 16487 tablet_service.cc:1467] Processing CreateTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 (DEFAULT_TABLE table=test-table [id=e36ef8070a524e4090772d2b4e79b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:13:50.764809 16487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 109da5d315a94e2d8783f9f39a2bcda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:13:50.768605 16574 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap starting.
I20260808 13:13:50.771777 16574 tablet_bootstrap.cc:654] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:50.772794 16574 log.cc:824] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:50.775681 16574 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: No bootstrap required, opened a new log
I20260808 13:13:50.776093 16574 ts_tablet_manager.cc:1397] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:13:50.777208 16576 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Bootstrap starting.
I20260808 13:13:50.778757 16577 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Bootstrap starting.
I20260808 13:13:50.780894 16576 tablet_bootstrap.cc:654] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:50.782090 16577 tablet_bootstrap.cc:654] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Neither blocks nor log segments found. Creating new log.
I20260808 13:13:50.782204 16576 log.cc:824] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:50.783150 16577 log.cc:824] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:13:50.785423 16576 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: No bootstrap required, opened a new log
I20260808 13:13:50.785586 16577 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: No bootstrap required, opened a new log
I20260808 13:13:50.785804 16576 ts_tablet_manager.cc:1397] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260808 13:13:50.785931 16577 ts_tablet_manager.cc:1397] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260808 13:13:50.787072 16574 raft_consensus.cc:348] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.787556 16574 raft_consensus.cc:374] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:50.787680 16574 raft_consensus.cc:729] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7e8b5e793b471cbd8aa9b6796187f6, State: Initialized, Role: FOLLOWER
I20260808 13:13:50.788335 16574 consensus_queue.cc:260] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.791709 16574 ts_tablet_manager.cc:1428] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260808 13:13:50.794468 16577 raft_consensus.cc:348] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.794863 16577 raft_consensus.cc:374] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:50.794976 16577 raft_consensus.cc:729] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8463024245f441629bee3e2615d8eb2d, State: Initialized, Role: FOLLOWER
I20260808 13:13:50.795027 16576 raft_consensus.cc:348] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.795491 16576 raft_consensus.cc:374] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:13:50.795588 16576 raft_consensus.cc:729] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bdc9121ff14c55ac3ca01080a537ab, State: Initialized, Role: FOLLOWER
I20260808 13:13:50.795601 16577 consensus_queue.cc:260] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.796121 16576 consensus_queue.cc:260] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.796671 16555 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:13:50.797145 16414 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:13:50.797710 16576 ts_tablet_manager.cc:1428] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260808 13:13:50.798458 16577 ts_tablet_manager.cc:1428] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Time spent starting tablet: real 0.012s user 0.003s sys 0.007s
W20260808 13:13:50.863451 16271 tablet.cc:2369] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:13:50.899746 16415 tablet.cc:2369] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:13:50.919512 16556 tablet.cc:2369] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:13:50.931267 16581 raft_consensus.cc:482] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:13:50.931559 16581 raft_consensus.cc:504] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.933293 16581 leader_election.cc:288] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
I20260808 13:13:50.939410 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.933755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39477 (local address 127.31.59.129:45387)
0808 13:13:50.933986 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.933992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.934186 (+ 194us) server_negotiation.cc:407] Connection header received
0808 13:13:50.934406 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.934412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.934498 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.934594 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:50.935041 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.935795 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.937380 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.938563 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.938723 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.938796 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.939061 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.939154 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 13:13:50.939488 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.933596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.31.59.131:39477)
0808 13:13:50.934075 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.934096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.934254 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.934634 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.934640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.934669 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:50.934894 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.934903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.935936 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.935942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.937212 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.937224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.938712 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.938719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.938909 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.938944 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.939045 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.939179 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338}
I20260808 13:13:50.940117 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.933854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:59783 (local address 127.31.59.130:34003)
0808 13:13:50.934057 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.934062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.934667 (+ 605us) server_negotiation.cc:407] Connection header received
0808 13:13:50.934857 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.934863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.934933 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.935016 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:13:50.935546 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.936230 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.937955 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.939159 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.939298 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.939369 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.939744 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.939872 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:13:50.940169 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.933800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.31.59.131:59783)
0808 13:13:50.934564 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:50.934582 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:13:50.934701 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:13:50.935041 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:13:50.935045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:13:50.935068 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:13:50.935400 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.935409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.936377 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.936381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.937802 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:13:50.937817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.939358 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.939369 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:13:50.939543 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.939575 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:13:50.939712 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 13:13:50.939846 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":410,"threads_started":1}
I20260808 13:13:50.940663 16220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "109da5d315a94e2d8783f9f39a2bcda0" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" is_pre_election: true
I20260808 13:13:50.941314 16220 raft_consensus.cc:2454] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 0.
I20260808 13:13:50.941291 16368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "109da5d315a94e2d8783f9f39a2bcda0" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" is_pre_election: true
I20260808 13:13:50.941879 16368 raft_consensus.cc:2454] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 0.
I20260808 13:13:50.942097 16439 leader_election.cc:302] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8463024245f441629bee3e2615d8eb2d, cd7e8b5e793b471cbd8aa9b6796187f6; no voters:
I20260808 13:13:50.942517 16581 raft_consensus.cc:2781] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:13:50.942668 16581 raft_consensus.cc:482] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:13:50.942790 16581 raft_consensus.cc:3037] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:50.944955 16581 raft_consensus.cc:504] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.945586 16581 leader_election.cc:288] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 election: Requested vote from peers cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
I20260808 13:13:50.946060 16368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "109da5d315a94e2d8783f9f39a2bcda0" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
I20260808 13:13:50.946053 16220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "109da5d315a94e2d8783f9f39a2bcda0" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
I20260808 13:13:50.946271 16368 raft_consensus.cc:3037] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:50.946281 16220 raft_consensus.cc:3037] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:13:50.949481 16220 raft_consensus.cc:2454] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 1.
I20260808 13:13:50.949482 16368 raft_consensus.cc:2454] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 1.
I20260808 13:13:50.949941 16438 leader_election.cc:302] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab; no voters:
I20260808 13:13:50.950275 16581 raft_consensus.cc:2781] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:13:50.950708 16581 raft_consensus.cc:686] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 1 LEADER]: Becoming Leader. State: Replica: 8463024245f441629bee3e2615d8eb2d, State: Running, Role: LEADER
I20260808 13:13:50.951177 16581 consensus_queue.cc:237] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:13:50.955893 16065 catalog_manager.cc:5495] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8463024245f441629bee3e2615d8eb2d (127.31.59.131). New cstate: current_term: 1 leader_uuid: "8463024245f441629bee3e2615d8eb2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } health_report { overall_health: UNKNOWN } } }
I20260808 13:13:50.985667 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:50.980485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.31.59.131:39335)
0808 13:13:50.980706 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:50.980710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:50.980729 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:13:50.980897 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:50.980900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:50.980982 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:50.981120 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:13:50.981432 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.982221 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.983689 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:50.984100 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:50.984200 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:50.984679 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:13:50.985192 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:13:50.985306 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:50.985347 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 13:13:50.985418 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 13:13:50.993686 16368 raft_consensus.cc:1261] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Refusing update from remote peer 8463024245f441629bee3e2615d8eb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:50.993686 16220 raft_consensus.cc:1261] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Refusing update from remote peer 8463024245f441629bee3e2615d8eb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:13:50.994585 16584 consensus_queue.cc:1035] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:50.994958 16581 consensus_queue.cc:1035] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:13:51.004719 16587 mvcc.cc:205] Tried to move back new op lower bound from 7316254027740356608 to 7316254027579559936. Current Snapshot: MvccSnapshot[applied={T|T < 7316254027740356608}]
I20260808 13:13:51.010237 16588 mvcc.cc:205] Tried to move back new op lower bound from 7316254027740356608 to 7316254027579559936. Current Snapshot: MvccSnapshot[applied={T|T < 7316254027740356608}]
I20260808 13:13:51.020419 16066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:51.020706 16066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:13:51.024592 16066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Sending DeleteTablet for 3 replicas of tablet 109da5d315a94e2d8783f9f39a2bcda0
I20260808 13:13:51.025810 16195 tablet_service.cc:1514] Processing DeleteTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:51 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:59690
I20260808 13:13:51.025974 16344 tablet_service.cc:1514] Processing DeleteTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:51 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:43734
I20260808 13:13:51.026579 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16129
I20260808 13:13:51.026607 16487 tablet_service.cc:1514] Processing DeleteTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:51 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:38964
Aug 08 13:13:51 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:13:51.050503 16602 tablet_replica.cc:330] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: stopping tablet replica
I20260808 13:13:51.051270 16602 raft_consensus.cc:2229] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:13:51.052184 16602 raft_consensus.cc:2258] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:51.052872 16600 tablet_replica.cc:330] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: stopping tablet replica
I20260808 13:13:51.053431 16600 raft_consensus.cc:2229] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:51.056172 16602 ts_tablet_manager.cc:1905] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:51.056823 16600 raft_consensus.cc:2258] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:51.060068 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.027807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.31.59.129:45387)
0808 13:13:51.027990 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:51.027996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:51.038746 (+ 10750us) server_negotiation.cc:407] Connection header received
0808 13:13:51.039253 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:51.039260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:51.039350 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:51.039427 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:51.041688 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.042917 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.044921 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.045592 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.045694 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:51.051631 (+ 5937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:51.051653 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:51.051671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:51.051741 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:51.056132 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.057101 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.057110 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.057114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.057520 (+ 406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.058018 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.058023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.058026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.058488 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:51.058788 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:51.059455 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 13:13:51.059689 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:13:51.062116 16265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:51.063867 16601 tablet_replica.cc:330] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: stopping tablet replica
I20260808 13:13:51.064451 16601 raft_consensus.cc:2229] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:13:51.065176 16601 raft_consensus.cc:2258] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:13:51.068653 16602 ts_tablet_manager.cc:1918] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:51.068842 16602 log.cc:1196] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/109da5d315a94e2d8783f9f39a2bcda0
I20260808 13:13:51.070039 16602 ts_tablet_manager.cc:1939] T 109da5d315a94e2d8783f9f39a2bcda0 P 8463024245f441629bee3e2615d8eb2d: Deleting consensus metadata
I20260808 13:13:51.070561 16601 ts_tablet_manager.cc:1905] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:51.072007 16052 catalog_manager.cc:4841] TS 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): tablet 109da5d315a94e2d8783f9f39a2bcda0 (table test-table [id=e36ef8070a524e4090772d2b4e79b9b3]) successfully deleted
I20260808 13:13:51.075819 16601 ts_tablet_manager.cc:1918] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:13:51.075959 16601 log.cc:1196] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/109da5d315a94e2d8783f9f39a2bcda0
I20260808 13:13:51.076380 16601 ts_tablet_manager.cc:1939] T 109da5d315a94e2d8783f9f39a2bcda0 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting consensus metadata
I20260808 13:13:51.077880 16051 catalog_manager.cc:4841] TS b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): tablet 109da5d315a94e2d8783f9f39a2bcda0 (table test-table [id=e36ef8070a524e4090772d2b4e79b9b3]) successfully deleted
I20260808 13:13:51.063014 16600 ts_tablet_manager.cc:1905] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:13:51.214704 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16129
W20260808 13:13:51.234606 16052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:45387 (error 108)
I20260808 13:13:51.234720 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16276
I20260808 13:13:51.236661 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.236117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59720)
0808 13:13:51.236348 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.236468 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 13:13:51.236961 16052 catalog_manager.cc:4584] TS cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 109da5d315a94e2d8783f9f39a2bcda0: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Aug 08 13:13:51 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:13:51.251120 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.235961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.31.59.130:34003)
0808 13:13:51.236174 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:51.236180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:51.236201 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:13:51.236267 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:51.236272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:51.236349 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:51.236449 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:51.237974 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.238754 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.240213 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.240631 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.240740 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:51.245200 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:51.245216 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:51.245231 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:51.245268 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:51.248107 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.248805 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.248813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.248815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.248937 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.249306 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.249310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.249312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.249880 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:51.250180 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:51.250481 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:13:51.250704 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:13:51.251812 16409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:51.290691 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.290135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59732)
0808 13:13:51.290396 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.290506 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:13:51.291880 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.291457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59748)
0808 13:13:51.291625 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.291731 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:13:51.344266 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16276
I20260808 13:13:51.352785 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.352230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59752)
0808 13:13:51.352484 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.352592 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 13:13:51.354064 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.353608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59760)
0808 13:13:51.353786 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.353886 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 13:13:51.364776 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16420
Aug 08 13:13:51 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:13:51.383061 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.365667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.31.59.131:39335)
0808 13:13:51.365886 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:13:51.365892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:13:51.365919 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:13:51.366025 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:13:51.366046 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:13:51.366132 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:13:51.366301 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:13:51.367530 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.368721 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.370264 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:13:51.370928 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:13:51.371077 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:13:51.375573 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:13:51.375597 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:13:51.375623 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:13:51.375694 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:13:51.379779 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.380490 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.380500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.380503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.380777 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:13:51.381198 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:13:51.381207 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:13:51.381210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:13:51.381774 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:13:51.382139 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0808 13:13:51.382369 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 13:13:51.382605 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:13:51.383824 16549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:51.434765 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.434276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59772)
0808 13:13:51.434495 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.434604 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 13:13:51.435943 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.435496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59778)
0808 13:13:51.435675 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.435790 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 13:13:51.511052 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16420
I20260808 13:13:51.533241 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16032
I20260808 13:13:51.534479 16097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:13:51.646296 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:13:51.645772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:59786)
0808 13:13:51.646011 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 13:13:51.646130 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:13:51.646548 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16032
I20260808 13:14:11.670231 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--webserver_interface=127.31.59.190
--webserver_port=39945
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:36101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:12.011785 16610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:12.012055 16610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:12.012204 16610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:12.019361 16610 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:14:12.019457 16610 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:14:12.019520 16610 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:14:12.019593 16610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:14:12.019665 16610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:12.019761 16610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:12.019819 16610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:14:12.019893 16610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:14:12.029831 16610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:36101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=39945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:12.031121 16610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:12.032516 16610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:12.057878 16621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:12.057837 16619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:12.057902 16617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:12.058394 16610 server_base.cc:1034] running on GCE node
I20260808 13:14:12.059307 16610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:12.060913 16610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:12.062230 16610 hybrid_clock.cc:648] HybridClock initialized: now 1786194852062137 us; error 38 us; skew 500 ppm
I20260808 13:14:12.062660 16610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:12 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:12.066538 16610 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:14:12.068277 16610 webserver.cc:460] Webserver started at http://127.31.59.190:39945/ using document root <none> and password file <none>
I20260808 13:14:12.069077 16610 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:12.069211 16610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:12.075523 16610 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:14:12.078545 16628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:12.079946 16610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:14:12.080117 16610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "150b8fe8a4704874bc8a4903bd44ec33"
format_stamp: "Formatted at 2026-08-08 13:13:48 on dist-test-slave-49h3"
server_key: "2ac3ea081b22bf89622396be2733c752"
server_key_iv: "51878987c84655377b20baba1a327118"
server_key_version: "encryptionkey@0"
I20260808 13:14:12.081215 16610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:12.123560 16610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:12.127871 16610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:12.128191 16610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:12.147073 16610 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:36101
I20260808 13:14:12.147107 16685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:36101 every 8 connection(s)
I20260808 13:14:12.148699 16610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:14:12.150913 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16610
I20260808 13:14:12.151926 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:45387
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=44501
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:14:12.157780 16686 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:14:12.162223 16686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap starting.
I20260808 13:14:12.183085 16686 log.cc:824] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:12.195298 16686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:14:12.195835 16686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap complete.
I20260808 13:14:12.205370 16686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:12.206354 16686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Initialized, Role: FOLLOWER
I20260808 13:14:12.206997 16686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:12.207237 16686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:14:12.207381 16686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:14:12.207556 16686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:14:12.211519 16686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:12.212039 16686 leader_election.cc:302] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150b8fe8a4704874bc8a4903bd44ec33; no voters:
I20260808 13:14:12.212757 16686 leader_election.cc:288] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:14:12.212899 16690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:14:12.214036 16690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 LEADER]: Becoming Leader. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Running, Role: LEADER
I20260808 13:14:12.214689 16690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:12.215124 16686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:14:12.217164 16692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8fe8a4704874bc8a4903bd44ec33. Latest consensus state: current_term: 2 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:14:12.217567 16692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:12.218132 16691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:14:12.218441 16691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:12.226933 16697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:14:12.235729 16697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:14:12.237572 16697 catalog_manager.cc:1259] Loaded cluster ID: d87a62e679bd4dd9b072d35c7f934483
I20260808 13:14:12.237747 16697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:14:12.242065 16697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:14:12.243779 16697 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Loaded TSK: 0
I20260808 13:14:12.273399 16707 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:14:12.279270 16697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Generated new TSK 1
I20260808 13:14:12.279662 16697 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:14:12.533335 16688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:12.533663 16688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:12.533836 16688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:12.541137 16688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:12.541249 16688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:12.541393 16688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:14:12.551594 16688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:45387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=44501
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:12.553017 16688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:12.554369 16688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:12.567528 16713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:12.567500 16715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:12.568493 16717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:12.569203 16688 server_base.cc:1034] running on GCE node
I20260808 13:14:12.570035 16688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:12.571372 16688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:12.572664 16688 hybrid_clock.cc:648] HybridClock initialized: now 1786194852572598 us; error 62 us; skew 500 ppm
I20260808 13:14:12.573123 16688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:12 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:12.578171 16688 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:14:12.580433 16688 webserver.cc:460] Webserver started at http://127.31.59.129:44501/ using document root <none> and password file <none>
I20260808 13:14:12.581358 16688 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:12.581573 16688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:12.588639 16688 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:14:12.592593 16724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:12.594238 16688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:14:12.594425 16688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
format_stamp: "Formatted at 2026-08-08 13:13:49 on dist-test-slave-49h3"
server_key: "e286a5944fc8e1f50cc4da5ed68daca5"
server_key_iv: "c6ffeb5bda6db3564f6f5ef721015f61"
server_key_version: "encryptionkey@0"
I20260808 13:14:12.595834 16688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:12.616597 16688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:12.622263 16688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:12.622573 16688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:12.623932 16688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:12.626356 16731 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 13:14:12.632015 16688 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 13:14:12.632160 16688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 13:14:12.632337 16688 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 13:14:12.637105 16688 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 13:14:12.637282 16688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.005s sys 0.000s
I20260808 13:14:12.637661 16731 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap starting.
I20260808 13:14:12.645741 16731 log.cc:824] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:12.659490 16731 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:14:12.660120 16731 tablet_bootstrap.cc:492] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap complete.
I20260808 13:14:12.660552 16731 ts_tablet_manager.cc:1397] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.000s
I20260808 13:14:12.670054 16731 raft_consensus.cc:348] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:14:12.671507 16731 raft_consensus.cc:729] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7e8b5e793b471cbd8aa9b6796187f6, State: Initialized, Role: FOLLOWER
I20260808 13:14:12.672210 16731 consensus_queue.cc:260] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } } peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } }
I20260808 13:14:12.673895 16731 ts_tablet_manager.cc:1428] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent starting tablet: real 0.013s user 0.016s sys 0.000s
I20260808 13:14:12.681095 16688 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45387
I20260808 13:14:12.681156 16847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45387 every 8 connection(s)
I20260808 13:14:12.682350 16688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:14:12.685310 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16688
I20260808 13:14:12.686044 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:34003
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=46585
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:14:12 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:12.709949 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:12.685079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:54143)
0808 13:14:12.690062 (+ 4983us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:12.690115 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:12.690670 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:12.692093 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:12.692100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:12.692703 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:12.693332 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:12.693361 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.695498 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.695501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:12.696733 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:12.696752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.698065 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.698073 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:12.698361 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:12.699082 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:12.699113 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:12.701611 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:12.705805 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:12.705812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:12.705825 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:12.706331 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:12.706862 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:12.706865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:12.706868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:12.707039 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:12.707895 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:12.707900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:12.708203 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 13:14:12.708431 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 13:14:12.708844 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4688,"thread_start_us":4481,"threads_started":1}
I20260808 13:14:12.710309 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:12.685166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54143 (local address 127.31.59.190:36101)
0808 13:14:12.685827 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:12.685845 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:12.690281 (+ 4436us) server_negotiation.cc:407] Connection header received
0808 13:14:12.691190 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:12.691208 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:12.691649 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:12.692029 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:12.693631 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.695352 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.696880 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.697831 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.697998 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:12.701883 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:12.701893 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:12.701912 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:12.701956 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:12.705628 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:12.706483 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:12.706491 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:12.706498 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:12.706763 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:12.707161 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:12.707166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:12.707168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:12.707787 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:12.707974 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:12.708718 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 13:14:12.709139 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":292,"threads_started":1}
I20260808 13:14:12.713162 16848 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:14:12.713521 16848 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:12.714252 16848 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:14:12.717808 16647 ts_manager.cc:194] Registered new tserver with Master: cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387)
I20260808 13:14:12.723115 16647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:54143
I20260808 13:14:12.740005 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:12.722262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60798)
0808 13:14:12.733230 (+ 10968us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:12.733279 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:12.733521 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:12.734162 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:12.734167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:12.734192 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:12.734452 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:12.734460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.736260 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.736263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:12.737649 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:12.737670 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.739345 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.739349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:12.739473 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:12.739507 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:14:12.739590 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:14:12.739716 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10736,"thread_start_us":10635,"threads_started":1}
I20260808 13:14:12.740159 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:12.722287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.31.59.129:45387)
0808 13:14:12.732906 (+ 10619us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:12.732915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:12.733487 (+ 572us) server_negotiation.cc:407] Connection header received
0808 13:14:12.733727 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:12.733735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:12.733963 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:12.734151 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:12.734599 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.736153 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.737817 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:12.739188 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:12.739325 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:12.739555 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:12.739640 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 13:14:12.739793 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10428,"thread_start_us":10269,"threads_started":1}
I20260808 13:14:12.741501 16779 tablet_service.cc:1514] Processing DeleteTablet for tablet 109da5d315a94e2d8783f9f39a2bcda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:13:51 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:60798
I20260808 13:14:12.743588 16857 tablet_replica.cc:330] stopping tablet replica
I20260808 13:14:12.743994 16857 raft_consensus.cc:2229] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:12.744365 16857 raft_consensus.cc:2258] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:12.745579 16857 ts_tablet_manager.cc:1905] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:12.752693 16857 ts_tablet_manager.cc:1918] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:12.752856 16857 log.cc:1196] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/109da5d315a94e2d8783f9f39a2bcda0
I20260808 13:14:12.753319 16857 ts_tablet_manager.cc:1939] T 109da5d315a94e2d8783f9f39a2bcda0 P cd7e8b5e793b471cbd8aa9b6796187f6: Deleting consensus metadata
I20260808 13:14:12.755527 16631 catalog_manager.cc:4841] TS cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387): tablet 109da5d315a94e2d8783f9f39a2bcda0 (table test-table [id=e36ef8070a524e4090772d2b4e79b9b3]) successfully deleted
W20260808 13:14:13.033574 16852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:13.033885 16852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:13.034036 16852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:13.041290 16852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:13.041414 16852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:13.041586 16852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:14:13.051417 16852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:34003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=46585
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:13.052712 16852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:13.054174 16852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:13.067049 16862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:13.067015 16863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:13.066979 16865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:13.067654 16852 server_base.cc:1034] running on GCE node
I20260808 13:14:13.068275 16852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:13.069175 16852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:13.070371 16852 hybrid_clock.cc:648] HybridClock initialized: now 1786194853070319 us; error 51 us; skew 500 ppm
I20260808 13:14:13.070672 16852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:13 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:13.073736 16852 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:14:13.075176 16852 webserver.cc:460] Webserver started at http://127.31.59.130:46585/ using document root <none> and password file <none>
I20260808 13:14:13.075845 16852 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:13.075978 16852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:13.081351 16852 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:14:13.083951 16873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.085271 16852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:14:13.085428 16852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "66a0502ef12b36fa4c46099bbe197f2f"
server_key_iv: "638d3af540400641b53af54023fe4a28"
server_key_version: "encryptionkey@0"
I20260808 13:14:13.086494 16852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:13.131888 16852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:13.136860 16852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:13.137180 16852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:13.138513 16852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:13.141037 16852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:13.141163 16852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.141335 16852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:13.141424 16852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.172320 16852 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34003
I20260808 13:14:13.172398 16996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34003 every 8 connection(s)
I20260808 13:14:13.173727 16852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:14:13.181695 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 16852
I20260808 13:14:13.182293 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:39335
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=34285
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:14:13 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:13.193589 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.176144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54063 (local address 127.31.59.190:36101)
0808 13:14:13.176322 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.176327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.177462 (+ 1135us) server_negotiation.cc:407] Connection header received
0808 13:14:13.179268 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.179274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.179366 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.179455 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:13.180889 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.181688 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.183242 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.183730 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.183854 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.187539 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:13.187544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:13.187547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:13.187579 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:13.190384 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.191174 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.191178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.191181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.191330 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.191760 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.191764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.191766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.192076 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:13.192231 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.192939 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.193182 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:14:13.194664 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.176592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:54063)
0808 13:14:13.177287 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.177329 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.179007 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.179682 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.179690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.180192 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:13.180727 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.180744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.181842 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.181846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.183101 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.183122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.183951 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.183957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.184200 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.184882 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:13.184909 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:13.187322 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:13.190531 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:13.190540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:13.190556 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:13.191054 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:13.191440 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:13.191445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:13.191449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:13.191649 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:13.192223 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:13.192230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:13.192571 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.192825 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.193391 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":274,"threads_started":1}
I20260808 13:14:13.196203 16997 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:14:13.196532 16997 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:13.197412 16997 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:14:13.199043 16647 ts_manager.cc:194] Registered new tserver with Master: b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
I20260808 13:14:13.200220 16647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:54063
W20260808 13:14:13.516669 17000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:13.516935 17000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:13.517030 17000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:13.524252 17000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:13.524360 17000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:13.524487 17000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:14:13.534307 17000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:39335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=34285
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:13.535609 17000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:13.536854 17000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:13.549574 17005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:13.549649 17008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:13.549672 17006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:13.550428 17000 server_base.cc:1034] running on GCE node
I20260808 13:14:13.551067 17000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:13.552053 17000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:13.553258 17000 hybrid_clock.cc:648] HybridClock initialized: now 1786194853553214 us; error 33 us; skew 500 ppm
I20260808 13:14:13.553593 17000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:13 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:13.556607 17000 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:14:13.558048 17000 webserver.cc:460] Webserver started at http://127.31.59.131:34285/ using document root <none> and password file <none>
I20260808 13:14:13.558686 17000 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:13.558815 17000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:13.563956 17000 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 13:14:13.566531 17015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.567775 17000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 13:14:13.567920 17000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8463024245f441629bee3e2615d8eb2d"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "0bb135859782ceb024f53550eba09fb6"
server_key_iv: "15fcb56d979ad9273a35821cdcdf100c"
server_key_version: "encryptionkey@0"
I20260808 13:14:13.568872 17000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:13.599366 17000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:13.603989 17000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:13.604269 17000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:13.605311 17000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:13.606654 17000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:13.606752 17000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.606850 17000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:13.606940 17000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:13.636068 17000 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:39335
I20260808 13:14:13.636133 17137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:39335 every 8 connection(s)
I20260808 13:14:13.637465 17000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:14:13.645846 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17000
Aug 08 13:14:13 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:13.657652 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.639872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:42839 (local address 127.31.59.190:36101)
0808 13:14:13.640039 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.640043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.641149 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 13:14:13.643005 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.643011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.643089 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.643177 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:13.644592 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.645326 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.647169 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.647682 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.647758 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.651605 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:13.651622 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:13.651626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:13.651659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:13.654391 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.655289 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.655293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.655295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.655418 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.655850 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.655854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.655858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.656207 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:13.656327 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.657064 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.657271 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:14:13.658555 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.640345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:42839)
0808 13:14:13.640998 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.641034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.642745 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.643412 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.643419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.643903 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:13.644431 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.644448 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.645503 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.645507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.646988 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.647005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.648019 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.648024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.648206 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.648808 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:13.648829 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:13.651386 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:13.654544 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:13.654554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:13.654572 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:13.655167 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:13.655518 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:13.655523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:13.655526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:13.655729 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:13.656329 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:13.656337 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:13.656691 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.656922 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.657385 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":257,"threads_started":1}
I20260808 13:14:13.659999 17138 heartbeater.cc:347] Connected to a master server at 127.31.59.190:36101
I20260808 13:14:13.660295 17138 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:13.661170 17138 heartbeater.cc:511] Master 127.31.59.190:36101 requested a full tablet report, sending...
I20260808 13:14:13.661274 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.646721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.31.59.190:36101)
0808 13:14:13.647155 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.647161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.647486 (+ 325us) server_negotiation.cc:407] Connection header received
0808 13:14:13.647642 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.647646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.647710 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.647796 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:13.649349 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.650305 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.652094 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.652709 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.652834 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.655633 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:13.655637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:13.655640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:13.655667 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:13.658411 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.659504 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.659508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.659511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.659645 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.660000 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.660003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.660005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.660283 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:13.660505 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.660839 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.661002 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":227,"threads_started":1}
I20260808 13:14:13.662724 16647 ts_manager.cc:194] Registered new tserver with Master: 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335)
I20260808 13:14:13.663511 16647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:42839
I20260808 13:14:13.665109 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:14:13.671293 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.666180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.31.59.190:36101)
0808 13:14:13.666348 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.666352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.666731 (+ 379us) server_negotiation.cc:407] Connection header received
0808 13:14:13.666947 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.666950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.667063 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.667183 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:14:13.667503 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.668154 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.669588 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.669994 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.670075 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.670519 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:14:13.670788 (+ 269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 13:14:13.671070 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34126: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 13:14:13.671386 17141 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34126: token expired
W20260808 13:14:13.671931 16559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.59.190:36101)
I20260808 13:14:13.684234 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.672473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34136 (local address 127.31.59.190:36101)
0808 13:14:13.672628 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.672634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.672651 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:14:13.672690 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.672693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.672742 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.672811 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:13.674151 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.674796 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.676158 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.676554 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.676646 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.679041 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:13.679045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:13.679048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:13.679075 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:13.681798 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.682387 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.682392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.682396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.682543 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:13.682878 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:13.682883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:13.682886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:13.683262 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:13.683492 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.683688 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.683913 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 13:14:13.688135 16647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:14:13.698405 16647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:14:13.710188 16779 tablet_service.cc:1467] Processing CreateTablet for tablet 481228e64b51461aa4fd903d13f9c310 (DEFAULT_TABLE table=test-table [id=2794a4f165c04768a59374b26728951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:13.711139 16779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481228e64b51461aa4fd903d13f9c310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:13.717890 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.708329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39335 (local address 127.0.0.1:36666)
0808 13:14:13.709204 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.709226 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.709384 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.711170 (+ 1786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.711175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.711202 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.711536 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.711549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.713712 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.713718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.715574 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.715589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.716945 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.716950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.717123 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.717159 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.717250 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.717379 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":2,"thread_start_us":550,"threads_started":1}
I20260808 13:14:13.718890 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.708694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.31.59.131:39335)
0808 13:14:13.710577 (+ 1883us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.710584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.710611 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:14:13.710697 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.710705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.710976 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.711179 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.711770 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.713555 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.715716 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.716764 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.716893 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.717122 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.718390 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.718561 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":1552,"threads_started":1}
I20260808 13:14:13.719316 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.708773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.0.0.1:36640)
0808 13:14:13.709915 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.709937 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.710034 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.712394 (+ 2360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.712399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.712427 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.712746 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.712757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.714517 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.714522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.716670 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.716685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.718474 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.718481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.718650 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.718687 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.718766 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.718889 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":23,"thread_start_us":889,"threads_started":1}
I20260808 13:14:13.720355 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.709404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.31.59.130:34003)
0808 13:14:13.711829 (+ 2425us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.711837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.711865 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:14:13.711968 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.711975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.712182 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.712377 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.712906 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.714369 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.716835 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.718229 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.719542 (+ 1313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.719724 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.719822 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.720020 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2278,"thread_start_us":2115,"threads_started":1}
I20260808 13:14:13.720561 17153 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap starting.
I20260808 13:14:13.722045 17153 tablet_bootstrap.cc:654] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:13.722857 17064 tablet_service.cc:1467] Processing CreateTablet for tablet 481228e64b51461aa4fd903d13f9c310 (DEFAULT_TABLE table=test-table [id=2794a4f165c04768a59374b26728951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:13.724270 17153 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: No bootstrap required, opened a new log
I20260808 13:14:13.724432 17153 ts_tablet_manager.cc:1397] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 13:14:13.724846 17064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481228e64b51461aa4fd903d13f9c310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:13.724833 16926 tablet_service.cc:1467] Processing CreateTablet for tablet 481228e64b51461aa4fd903d13f9c310 (DEFAULT_TABLE table=test-table [id=2794a4f165c04768a59374b26728951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:13.725286 17153 raft_consensus.cc:348] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.725575 17153 raft_consensus.cc:374] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:13.725690 17153 raft_consensus.cc:729] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7e8b5e793b471cbd8aa9b6796187f6, State: Initialized, Role: FOLLOWER
I20260808 13:14:13.726044 16926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481228e64b51461aa4fd903d13f9c310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:13.725942 17153 consensus_queue.cc:260] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.726800 17153 ts_tablet_manager.cc:1428] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 13:14:13.726833 16848 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:14:13.734320 17155 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:13.734642 17155 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.743569 17155 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
I20260808 13:14:13.745491 17158 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Bootstrap starting.
I20260808 13:14:13.746371 17159 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Bootstrap starting.
I20260808 13:14:13.751479 17159 tablet_bootstrap.cc:654] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:13.751719 17158 tablet_bootstrap.cc:654] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:13.752961 17159 log.cc:824] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:13.753010 17158 log.cc:824] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:13.754717 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.737415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:56935 (local address 127.31.59.131:39335)
0808 13:14:13.737972 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.737978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.747241 (+ 9263us) server_negotiation.cc:407] Connection header received
0808 13:14:13.747416 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.747421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.747491 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.747572 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.748387 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.749576 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.752226 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.753488 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.753639 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.753721 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.754281 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.754427 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
I20260808 13:14:13.755512 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.736608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39335 (local address 127.31.59.129:56935)
0808 13:14:13.747136 (+ 10528us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.747152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.747294 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.747778 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.747783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.747816 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.748221 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.748234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.749788 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.749793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.752028 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.752049 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.753896 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.753903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.754094 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.754130 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.754962 (+ 832us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.755113 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10404,"mutex_wait_us":27,"thread_start_us":11047,"threads_started":1}
I20260808 13:14:13.756497 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.737711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:55867 (local address 127.31.59.130:34003)
0808 13:14:13.738108 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.738113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.748058 (+ 9945us) server_negotiation.cc:407] Connection header received
0808 13:14:13.748191 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.748196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.748267 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.748353 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.749132 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.750257 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.753945 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.755293 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.755417 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.755487 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.756077 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.756225 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260808 13:14:13.757368 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.736788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.31.59.129:55867)
0808 13:14:13.747946 (+ 11158us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.747967 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.748080 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.748525 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.748530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.748556 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.748939 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.748953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.751770 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.751774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.753739 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.753758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.755694 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.755701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.755894 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.755926 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.756811 (+ 885us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.756970 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11040,"mutex_wait_us":2,"thread_start_us":10576,"threads_started":1}
I20260808 13:14:13.756775 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8463024245f441629bee3e2615d8eb2d" is_pre_election: true
I20260808 13:14:13.758440 16947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" is_pre_election: true
W20260808 13:14:13.762315 16727 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:14:13.762583 16726 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:14:13.762735 16726 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:13.763165 17155 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:14:13.764659 17158 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: No bootstrap required, opened a new log
I20260808 13:14:13.764971 17158 ts_tablet_manager.cc:1397] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.004s
I20260808 13:14:13.765232 17159 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: No bootstrap required, opened a new log
I20260808 13:14:13.765527 17159 ts_tablet_manager.cc:1397] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.006s
I20260808 13:14:13.772305 17159 raft_consensus.cc:348] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.772305 17158 raft_consensus.cc:348] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.772802 17159 raft_consensus.cc:374] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:13.772791 17158 raft_consensus.cc:374] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:13.772900 17158 raft_consensus.cc:729] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bdc9121ff14c55ac3ca01080a537ab, State: Initialized, Role: FOLLOWER
I20260808 13:14:13.772934 17159 raft_consensus.cc:729] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8463024245f441629bee3e2615d8eb2d, State: Initialized, Role: FOLLOWER
I20260808 13:14:13.773653 17159 consensus_queue.cc:260] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.773653 17158 consensus_queue.cc:260] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.774876 17138 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:14:13.775607 17159 ts_tablet_manager.cc:1428] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260808 13:14:13.776572 16997 heartbeater.cc:503] Master 127.31.59.190:36101 was elected leader, sending a full tablet report...
I20260808 13:14:13.776954 17158 ts_tablet_manager.cc:1428] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260808 13:14:13.889024 17139 tablet.cc:2369] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:14:13.925913 16998 tablet.cc:2369] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:14:13.935282 16849 tablet.cc:2369] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:14:13.979368 17162 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:13.979728 17162 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.981483 17162 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003), cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387)
I20260808 13:14:13.988250 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.981785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.31.59.131:50921)
0808 13:14:13.982096 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.982122 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.982278 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.982864 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.982869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.982896 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.983042 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.983050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.984389 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.984393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.986386 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.986401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.987700 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.987704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.987848 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.987873 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.987943 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.988037 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 13:14:13.988406 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.981858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:50921 (local address 127.31.59.130:34003)
0808 13:14:13.982054 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.982060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.982206 (+ 146us) server_negotiation.cc:407] Connection header received
0808 13:14:13.982513 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.982520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.982607 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.982687 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.983181 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.984220 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.986532 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.987546 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.987711 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.987787 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.987984 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.988118 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:14:13.989077 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.981999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.31.59.131:55763)
0808 13:14:13.982693 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:13.982710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:13.982840 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:13.983336 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:13.983339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:13.983362 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:13.983675 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.983683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.984924 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.984929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.986848 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:13.986859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.988407 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.988413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:13.988564 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.988596 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:14:13.988692 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 13:14:13.988814 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":387,"threads_started":1}
I20260808 13:14:13.989177 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:13.982193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:55763 (local address 127.31.59.129:45387)
0808 13:14:13.983027 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:13.983031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:13.983049 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:14:13.983107 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:13.983111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:13.983182 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:13.983289 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:13.983862 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.984788 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.987006 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:13.988211 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:13.988391 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:13.988461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:13.988741 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 13:14:13.988887 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":575,"threads_started":1}
I20260808 13:14:13.989168 16947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" is_pre_election: true
I20260808 13:14:13.989635 16947 raft_consensus.cc:2454] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 0.
I20260808 13:14:13.990195 17017 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab; no voters:
I20260808 13:14:13.990110 16800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" is_pre_election: true
I20260808 13:14:13.990512 16800 raft_consensus.cc:2454] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 0.
I20260808 13:14:13.990629 17162 raft_consensus.cc:2781] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:14:13.990782 17162 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:14:13.990892 17162 raft_consensus.cc:3037] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:13.995720 17162 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:13.996379 17162 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 election: Requested vote from peers b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003), cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387)
I20260808 13:14:13.996941 16947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
I20260808 13:14:13.996951 16800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481228e64b51461aa4fd903d13f9c310" candidate_uuid: "8463024245f441629bee3e2615d8eb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
I20260808 13:14:13.997185 16800 raft_consensus.cc:3037] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:13.997172 16947 raft_consensus.cc:3037] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:13.999346 16947 raft_consensus.cc:2454] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 1.
I20260808 13:14:13.999473 16800 raft_consensus.cc:2454] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8463024245f441629bee3e2615d8eb2d in term 1.
I20260808 13:14:13.999818 17017 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab; no voters:
I20260808 13:14:14.000285 17162 raft_consensus.cc:2781] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:14:14.000810 17162 raft_consensus.cc:686] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 1 LEADER]: Becoming Leader. State: Replica: 8463024245f441629bee3e2615d8eb2d, State: Running, Role: LEADER
I20260808 13:14:14.001379 17162 consensus_queue.cc:237] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:14.006059 16647 catalog_manager.cc:5495] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8463024245f441629bee3e2615d8eb2d (127.31.59.131). New cstate: current_term: 1 leader_uuid: "8463024245f441629bee3e2615d8eb2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260808 13:14:14.088194 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.082530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.31.59.131:39335)
0808 13:14:14.082718 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:14.082724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:14.082822 (+ 98us) server_negotiation.cc:407] Connection header received
0808 13:14:14.083061 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:14.083065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:14.083143 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:14.083283 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:14:14.083674 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.084468 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.086011 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.086444 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.086553 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:14.087063 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:14:14.087577 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:14:14.087716 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:14.087816 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 13:14:14.087901 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:14:14.096436 16947 raft_consensus.cc:1261] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Refusing update from remote peer 8463024245f441629bee3e2615d8eb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:14.096460 16800 raft_consensus.cc:1261] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Refusing update from remote peer 8463024245f441629bee3e2615d8eb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:14.097365 17162 consensus_queue.cc:1035] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:14.097718 17166 consensus_queue.cc:1035] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:14.106829 17169 mvcc.cc:205] Tried to move back new op lower bound from 7316254122369187840 to 7316254121993543680. Current Snapshot: MvccSnapshot[applied={T|T < 7316254122369187840}]
I20260808 13:14:14.109560 17168 mvcc.cc:205] Tried to move back new op lower bound from 7316254122369187840 to 7316254121993543680. Current Snapshot: MvccSnapshot[applied={T|T < 7316254122369187840}]
I20260808 13:14:14.110913 17170 mvcc.cc:205] Tried to move back new op lower bound from 7316254122369187840 to 7316254121993543680. Current Snapshot: MvccSnapshot[applied={T|T < 7316254122369187840}]
I20260808 13:14:14.125213 16647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:14.125509 16647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:14.129061 16647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Sending DeleteTablet for 3 replicas of tablet 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:14.130149 17064 tablet_service.cc:1514] Processing DeleteTablet for tablet 481228e64b51461aa4fd903d13f9c310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36666
I20260808 13:14:14.130236 16926 tablet_service.cc:1514] Processing DeleteTablet for tablet 481228e64b51461aa4fd903d13f9c310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:36640
I20260808 13:14:14.130764 16779 tablet_service.cc:1514] Processing DeleteTablet for tablet 481228e64b51461aa4fd903d13f9c310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:14 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:60798
I20260808 13:14:14.131129 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16688
I20260808 13:14:14.142521 17180 tablet_replica.cc:330] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: stopping tablet replica
I20260808 13:14:14.142745 17180 raft_consensus.cc:2229] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:14.142856 17179 tablet_replica.cc:330] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: stopping tablet replica
I20260808 13:14:14.143108 17180 raft_consensus.cc:2258] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:14.143360 17179 raft_consensus.cc:2229] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:14.143947 17179 raft_consensus.cc:2258] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:14.144665 17180 ts_tablet_manager.cc:1905] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:14.144790 17178 tablet_replica.cc:330] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: stopping tablet replica
I20260808 13:14:14.145180 17178 raft_consensus.cc:2229] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:14:14.145961 17178 raft_consensus.cc:2258] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:14.148118 17179 ts_tablet_manager.cc:1905] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:14.149391 17178 ts_tablet_manager.cc:1905] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:14.157013 17178 ts_tablet_manager.cc:1918] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:14.162091 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.132854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.31.59.129:45387)
0808 13:14:14.133054 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:14.133060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:14.133078 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:14:14.133135 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:14.133139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:14.133223 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:14.133321 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:14.135384 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.136541 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.138510 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.139137 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.139257 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:14.148769 (+ 9512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:14.148791 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:14.148809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:14.148857 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:14.157449 (+ 8592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.158529 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.158539 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.158543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.158705 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.159174 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.159180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.159183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.159758 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:14.160067 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:14.161429 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0808 13:14:14.161749 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:14:14.163156 16843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:14.164157 17178 log.cc:1196] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/481228e64b51461aa4fd903d13f9c310
I20260808 13:14:14.164901 17178 ts_tablet_manager.cc:1939] T 481228e64b51461aa4fd903d13f9c310 P 8463024245f441629bee3e2615d8eb2d: Deleting consensus metadata
I20260808 13:14:14.167001 16631 catalog_manager.cc:4841] TS 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): tablet 481228e64b51461aa4fd903d13f9c310 (table test-table [id=2794a4f165c04768a59374b26728951f]) successfully deleted
I20260808 13:14:14.168352 17179 ts_tablet_manager.cc:1918] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:14.168553 17179 log.cc:1196] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/481228e64b51461aa4fd903d13f9c310
I20260808 13:14:14.169166 17179 ts_tablet_manager.cc:1939] T 481228e64b51461aa4fd903d13f9c310 P b1bdc9121ff14c55ac3ca01080a537ab: Deleting consensus metadata
I20260808 13:14:14.171263 16630 catalog_manager.cc:4841] TS b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): tablet 481228e64b51461aa4fd903d13f9c310 (table test-table [id=2794a4f165c04768a59374b26728951f]) successfully deleted
I20260808 13:14:14.274785 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16688
W20260808 13:14:14.303139 16631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:45387 (error 108)
I20260808 13:14:14.303331 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16852
I20260808 13:14:14.306360 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.304884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60824)
0808 13:14:14.305984 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.306134 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":879,"threads_started":1}
W20260808 13:14:14.306752 16631 catalog_manager.cc:4584] TS cd7e8b5e793b471cbd8aa9b6796187f6 (127.31.59.129:45387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 481228e64b51461aa4fd903d13f9c310: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
I20260808 13:14:14.318051 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.304135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.31.59.130:34003)
0808 13:14:14.304311 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:14.304317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:14.304337 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:14:14.304545 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:14.304551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:14.304651 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:14.304766 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:14.306501 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.307394 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.308769 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.309211 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.309322 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:14.312023 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:14.312039 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:14.312053 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:14.312088 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:14.314925 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.315673 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.315682 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.315685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.315820 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.316320 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.316326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.316330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.316854 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:14.317175 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:14.317420 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 13:14:14.317702 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:14:14.318753 16992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:14.360530 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.359936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60826)
0808 13:14:14.360198 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.360329 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 13:14:14.361810 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.361330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60832)
0808 13:14:14.361512 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.361618 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:14:14.411500 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16852
I20260808 13:14:14.422444 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.421963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60838)
0808 13:14:14.422194 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.422285 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:14:14.423458 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.423085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60852)
0808 13:14:14.423239 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.423329 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 13:14:14.440411 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17000
I20260808 13:14:14.458107 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.441283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.31.59.131:39335)
0808 13:14:14.441554 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:14.441560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:14.441839 (+ 279us) server_negotiation.cc:407] Connection header received
0808 13:14:14.442039 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:14.442043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:14.442120 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:14.442206 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:14.444137 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.444956 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.446991 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:14.447452 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:14.447561 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:14.451063 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:14.451078 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:14.451095 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:14.451142 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:14.454797 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.455780 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.455792 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.455794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.455936 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:14.456476 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:14.456481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:14.456485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:14.456934 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:14.457206 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:14.457629 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 13:14:14.457816 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:14:14.458802 17133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:14.504460 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.503959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60864)
0808 13:14:14.504199 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.504296 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 13:14:14.505757 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:14.505255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:45387 (local address 127.0.0.1:60868)
0808 13:14:14.505514 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:14.505614 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 13:14:14.584475 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17000
I20260808 13:14:14.606210 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 16610
I20260808 13:14:14.607400 16681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:14.722280 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 16610
I20260808 13:14:34.746210 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--webserver_interface=127.31.59.190
--webserver_port=39945
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:36101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:35.092883 17188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:35.093197 17188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:35.093298 17188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:35.100719 17188 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:14:35.100826 17188 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:14:35.100862 17188 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:14:35.100910 17188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:14:35.100980 17188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:35.101048 17188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:35.101094 17188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:14:35.101146 17188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:14:35.111203 17188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:36101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=39945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:35.112483 17188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:35.113802 17188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:35.125375 17194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:35.125298 17195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:35.125336 17197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:35.126735 17188 server_base.cc:1034] running on GCE node
I20260808 13:14:35.127563 17188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:35.128969 17188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:35.130211 17188 hybrid_clock.cc:648] HybridClock initialized: now 1786194875130128 us; error 42 us; skew 500 ppm
I20260808 13:14:35.130539 17188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:35 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194875, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:35.133625 17188 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:14:35.135013 17188 webserver.cc:460] Webserver started at http://127.31.59.190:39945/ using document root <none> and password file <none>
I20260808 13:14:35.135680 17188 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:35.135767 17188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:35.141079 17188 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:14:35.143947 17204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:35.145222 17188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:14:35.145366 17188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "150b8fe8a4704874bc8a4903bd44ec33"
format_stamp: "Formatted at 2026-08-08 13:13:48 on dist-test-slave-49h3"
server_key: "2ac3ea081b22bf89622396be2733c752"
server_key_iv: "51878987c84655377b20baba1a327118"
server_key_version: "encryptionkey@0"
I20260808 13:14:35.146394 17188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:35.163786 17188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:35.167914 17188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:35.168164 17188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:35.186233 17188 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:36101
I20260808 13:14:35.186280 17259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:36101 every 8 connection(s)
I20260808 13:14:35.187716 17188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:14:35.189482 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17188
I20260808 13:14:35.190551 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:45387
--local_ip_for_outbound_sockets=127.31.59.129
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=44501
--webserver_interface=127.31.59.129
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:14:35.196919 17260 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:14:35.201423 17260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap starting.
I20260808 13:14:35.223304 17260 log.cc:824] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:35.239848 17260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:14:35.240381 17260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Bootstrap complete.
I20260808 13:14:35.249693 17260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:35.250664 17260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Initialized, Role: FOLLOWER
I20260808 13:14:35.251341 17260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:35.251562 17260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:14:35.251675 17260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:14:35.251847 17260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:14:35.255734 17260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:35.256222 17260 leader_election.cc:302] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150b8fe8a4704874bc8a4903bd44ec33; no voters:
I20260808 13:14:35.256893 17260 leader_election.cc:288] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 13:14:35.257028 17264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:14:35.258124 17264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [term 3 LEADER]: Becoming Leader. State: Replica: 150b8fe8a4704874bc8a4903bd44ec33, State: Running, Role: LEADER
I20260808 13:14:35.258770 17264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } }
I20260808 13:14:35.259102 17260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:14:35.261538 17265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:14:35.261853 17265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:35.261226 17266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8fe8a4704874bc8a4903bd44ec33. Latest consensus state: current_term: 3 leader_uuid: "150b8fe8a4704874bc8a4903bd44ec33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150b8fe8a4704874bc8a4903bd44ec33" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 36101 } } }
I20260808 13:14:35.262494 17266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:35.270958 17273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:14:35.280192 17273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:14:35.281905 17273 catalog_manager.cc:1259] Loaded cluster ID: d87a62e679bd4dd9b072d35c7f934483
I20260808 13:14:35.282120 17273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:14:35.286542 17273 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:14:35.286939 17285 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:14:35.288218 17273 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Loaded TSK: 0
I20260808 13:14:35.288352 17273 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 150b8fe8a4704874bc8a4903bd44ec33: Loaded TSK: 1
F20260808 13:14:35.289157 17273 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa4f00bc9ad google::LogMessage::Fail() at ??:0
@ 0x7fa4f00bf88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa4f00bc4de google::LogMessage::Flush() at ??:0
@ 0x7fa4f00bda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa4f3e207d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa4f3e0f73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa505aeb1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa505aea783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa505b762a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fa505b760c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fa505bc05e9 std::function<>::operator()() at ??:0
@ 0x7fa505aeecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa505aecc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa505b74610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa505b74439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa4f2d08345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa4f2d0b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa4f2d0b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa4f2ce6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa4f5fc06db start_thread at ??:0
@ 0x7fa4eef5661f clone at ??:0
W20260808 13:14:35.597947 17262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:35.598222 17262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:35.598317 17262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:35.605608 17262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:35.605721 17262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:35.605842 17262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:14:35.615393 17262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:45387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=44501
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:35.616613 17262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:35.617983 17262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:35.630797 17314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:35.630833 17312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:35.631017 17311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:35.631685 17262 server_base.cc:1034] running on GCE node
I20260808 13:14:35.632277 17262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:35.633174 17262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:35.634402 17262 hybrid_clock.cc:648] HybridClock initialized: now 1786194875634346 us; error 43 us; skew 500 ppm
I20260808 13:14:35.634755 17262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:35 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194875, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:35.637861 17262 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:14:35.639309 17262 webserver.cc:460] Webserver started at http://127.31.59.129:44501/ using document root <none> and password file <none>
I20260808 13:14:35.639978 17262 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:35.640102 17262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:35.645535 17262 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:14:35.648509 17321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:35.649855 17262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:14:35.650000 17262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "cd7e8b5e793b471cbd8aa9b6796187f6"
format_stamp: "Formatted at 2026-08-08 13:13:49 on dist-test-slave-49h3"
server_key: "e286a5944fc8e1f50cc4da5ed68daca5"
server_key_iv: "c6ffeb5bda6db3564f6f5ef721015f61"
server_key_version: "encryptionkey@0"
I20260808 13:14:35.650935 17262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:35.667521 17262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:35.671684 17262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:35.671948 17262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:35.672938 17262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:35.674760 17330 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 13:14:35.680334 17262 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 13:14:35.680452 17262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 13:14:35.680565 17262 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 13:14:35.683905 17262 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 13:14:35.683987 17262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260808 13:14:35.684317 17330 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap starting.
I20260808 13:14:35.692830 17330 log.cc:824] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:35.708166 17330 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:14:35.709002 17330 tablet_bootstrap.cc:492] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Bootstrap complete.
I20260808 13:14:35.709611 17330 ts_tablet_manager.cc:1397] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent bootstrapping tablet: real 0.026s user 0.017s sys 0.008s
I20260808 13:14:35.717762 17330 raft_consensus.cc:348] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:35.719280 17330 raft_consensus.cc:729] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7e8b5e793b471cbd8aa9b6796187f6, State: Initialized, Role: FOLLOWER
I20260808 13:14:35.720069 17330 consensus_queue.cc:260] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:35.721679 17330 ts_tablet_manager.cc:1428] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260808 13:14:35.725610 17262 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:45387
I20260808 13:14:35.725679 17444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:45387 every 8 connection(s)
I20260808 13:14:35.726874 17262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:14:35.731448 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:35.729566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:43813)
0808 13:14:35.730333 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:35.730577 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":294,"threads_started":1}
W20260808 13:14:35.732134 17445 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:36101 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:36101: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
I20260808 13:14:35.732966 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:35.732646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:60781)
0808 13:14:35.732789 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:35.732861 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 13:14:35.733883 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:35.733533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:50233)
0808 13:14:35.733695 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:35.733768 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 13:14:35.734102 17445 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:14:35.735818 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17262
I20260808 13:14:35.736456 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:34003
--local_ip_for_outbound_sockets=127.31.59.130
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=46585
--webserver_interface=127.31.59.130
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:36.080708 17448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:36.080997 17448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:36.081103 17448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:36.088424 17448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:36.088534 17448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:36.088655 17448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:14:36.098816 17448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:34003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=46585
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:36.100180 17448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:36.101538 17448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:36.114396 17453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:36.114493 17454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:36.114493 17456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:36.115790 17448 server_base.cc:1034] running on GCE node
I20260808 13:14:36.116426 17448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:36.117378 17448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:36.118628 17448 hybrid_clock.cc:648] HybridClock initialized: now 1786194876118551 us; error 47 us; skew 500 ppm
I20260808 13:14:36.118940 17448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:36 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194876, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:36.122093 17448 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:14:36.123481 17448 webserver.cc:460] Webserver started at http://127.31.59.130:46585/ using document root <none> and password file <none>
I20260808 13:14:36.124153 17448 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:36.124277 17448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:36.129741 17448 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:14:36.132604 17464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.133998 17448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:14:36.134157 17448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "b1bdc9121ff14c55ac3ca01080a537ab"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "66a0502ef12b36fa4c46099bbe197f2f"
server_key_iv: "638d3af540400641b53af54023fe4a28"
server_key_version: "encryptionkey@0"
I20260808 13:14:36.135140 17448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:36.175403 17448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:36.179343 17448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:36.179615 17448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:36.180612 17448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:36.181962 17448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:36.182063 17448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.182170 17448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:36.182250 17448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.213995 17448 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34003
I20260808 13:14:36.214092 17588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34003 every 8 connection(s)
I20260808 13:14:36.215667 17448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:14:36.217770 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17448
I20260808 13:14:36.218559 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:39335
--local_ip_for_outbound_sockets=127.31.59.131
--tserver_master_addrs=127.31.59.190:36101
--webserver_port=34285
--webserver_interface=127.31.59.131
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:14:36.221005 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.219026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:38057)
0808 13:14:36.219825 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.220117 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":315,"threads_started":1}
W20260808 13:14:36.221762 17589 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:36101 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:36101: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
I20260808 13:14:36.222689 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.222332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:39359)
0808 13:14:36.222506 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.222578 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 13:14:36.223528 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.223223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:54735)
0808 13:14:36.223344 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.223420 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 13:14:36.223771 17589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:14:36.557106 17592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:36.557405 17592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:36.557550 17592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:36.565364 17592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:36.565546 17592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:36.565739 17592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:14:36.575889 17592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:39335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=34285
--tserver_master_addrs=127.31.59.190:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:36.577248 17592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:36.578629 17592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:36.591213 17597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:36.591267 17600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:36.591305 17598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:36.592465 17592 server_base.cc:1034] running on GCE node
I20260808 13:14:36.593051 17592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:36.594010 17592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:36.595211 17592 hybrid_clock.cc:648] HybridClock initialized: now 1786194876595164 us; error 35 us; skew 500 ppm
I20260808 13:14:36.595517 17592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:36 dist-test-slave-49h3 krb5kdc[16018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194876, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:36.598635 17592 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:14:36.600219 17592 webserver.cc:460] Webserver started at http://127.31.59.131:34285/ using document root <none> and password file <none>
I20260808 13:14:36.600888 17592 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:36.601009 17592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:36.606343 17592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:14:36.608914 17608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.610212 17592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:14:36.610369 17592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "8463024245f441629bee3e2615d8eb2d"
format_stamp: "Formatted at 2026-08-08 13:13:50 on dist-test-slave-49h3"
server_key: "0bb135859782ceb024f53550eba09fb6"
server_key_iv: "15fcb56d979ad9273a35821cdcdf100c"
server_key_version: "encryptionkey@0"
I20260808 13:14:36.611362 17592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:36.641844 17592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:36.646395 17592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:36.646674 17592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:36.647737 17592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:36.649163 17592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:36.649263 17592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.649379 17592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:36.649469 17592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:36.680904 17592 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:39335
I20260808 13:14:36.680974 17728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:39335 every 8 connection(s)
I20260808 13:14:36.682374 17592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:14:36.686945 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.685093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:36645)
0808 13:14:36.685843 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.686104 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":305,"threads_started":1}
W20260808 13:14:36.687610 17729 heartbeater.cc:644] Failed to heartbeat to 127.31.59.190:36101 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.190:36101: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
I20260808 13:14:36.688400 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.688074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:36175)
0808 13:14:36.688228 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.688295 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:14:36.688963 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17592
I20260808 13:14:36.689296 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.688968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:40071)
0808 13:14:36.689095 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.689187 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:14:36.689550 17729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:14:36.735452 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:36.734638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:41311)
0808 13:14:36.735139 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:36.735233 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":247,"threads_started":1}
I20260808 13:14:37.180976 17735 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:37.181298 17735 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:37.182960 17735 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
Aug 08 13:14:37 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194875, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
Aug 08 13:14:37 dist-test-slave-49h3 krb5kdc[16018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194875, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:14:37.205502 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.183408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:39335 (local address 127.31.59.129:36165)
0808 13:14:37.185638 (+ 2230us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:37.185655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:37.185791 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:37.188098 (+ 2307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:37.188101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:37.188636 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:37.189381 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:37.189400 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.191671 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.191676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:37.193252 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:37.193265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.194272 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.194279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:37.194566 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:37.195380 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:37.195397 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:37.197722 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:37.201261 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:37.201271 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:37.201286 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:37.201876 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:37.202367 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:37.202375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:37.202640 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:37.203629 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:37.203637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:37.204121 (+ 484us) client_negotiation.cc:769] Sending connection context
0808 13:14:37.204445 (+ 324us) client_negotiation.cc:240] Negotiation successful
0808 13:14:37.204991 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2093,"mutex_wait_us":431,"spinlock_wait_cycles":28416,"thread_start_us":1687,"threads_started":1}
I20260808 13:14:37.205569 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.183656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:36165 (local address 127.31.59.131:39335)
0808 13:14:37.185309 (+ 1653us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:37.185327 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:37.185766 (+ 439us) server_negotiation.cc:407] Connection header received
0808 13:14:37.187375 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:37.187384 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:37.187777 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:37.188121 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:37.189645 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.191454 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.193392 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.194076 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.194267 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:37.198003 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:37.198011 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:37.198025 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:37.198061 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:37.201119 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202009 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:37.202016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:37.202022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:37.202244 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202781 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:37.202787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:37.202790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:37.203474 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:37.203666 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:37.204719 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0808 13:14:37.205114 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":1344,"threads_started":1}
I20260808 13:14:37.205569 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.183588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:40749 (local address 127.31.59.130:34003)
0808 13:14:37.185013 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:37.185029 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:37.185072 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:14:37.186906 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:37.186916 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:37.187295 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:37.187546 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:37.189580 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.191454 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.193416 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.194106 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.194285 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:37.198246 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:37.198253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:37.198265 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:37.198302 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:37.201210 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202069 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:37.202077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:37.202085 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:37.202262 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202782 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:37.202787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:37.202790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:37.203474 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:37.203668 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:37.204705 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0808 13:14:37.205114 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":1076,"threads_started":1}
I20260808 13:14:37.205514 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.183399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34003 (local address 127.31.59.129:40749)
0808 13:14:37.183569 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:37.183618 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:37.184210 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:37.187756 (+ 3546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:37.187768 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:37.188652 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:37.189381 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:37.189401 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.191680 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.191684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:37.193250 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:37.193263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:37.194335 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:37.194342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:37.194575 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:37.195306 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:37.195333 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:37.197999 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:37.201335 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:37.201341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:37.201345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:37.201908 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:37.202379 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:37.202383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:37.202387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:37.202640 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:37.203629 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:37.203637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:37.204122 (+ 485us) client_negotiation.cc:769] Sending connection context
0808 13:14:37.204445 (+ 323us) client_negotiation.cc:240] Negotiation successful
0808 13:14:37.205124 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":496,"spinlock_wait_cycles":19328}
W20260808 13:14:37.209481 17325 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
W20260808 13:14:37.209786 17324 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:37.209929 17324 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:37.210327 17735 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:14:37.225296 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.224334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:57749)
0808 13:14:37.224854 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:37.225026 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":242,"threads_started":1}
I20260808 13:14:37.690969 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.690085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:48573)
0808 13:14:37.690590 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:37.690716 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":245,"threads_started":1}
I20260808 13:14:37.736991 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:37.736321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:58433)
0808 13:14:37.736724 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:37.736811 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":201,"threads_started":1}
I20260808 13:14:38.226881 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:38.226177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:40415)
0808 13:14:38.226623 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:38.226717 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":225,"threads_started":1}
I20260808 13:14:38.692541 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:38.691858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:41089)
0808 13:14:38.692289 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:38.692372 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":217,"threads_started":1}
I20260808 13:14:38.738689 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:38.737869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:46593)
0808 13:14:38.738408 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:38.738498 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":251,"threads_started":1}
I20260808 13:14:39.129076 17745 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:39.129297 17745 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:39.129989 17745 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
W20260808 13:14:39.131024 17324 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
W20260808 13:14:39.131206 17325 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:39.131318 17325 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:39.131556 17745 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:14:39.228428 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:39.227672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:57939)
0808 13:14:39.228167 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:39.228267 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":253,"threads_started":1}
I20260808 13:14:39.694016 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:39.693297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:39115)
0808 13:14:39.693766 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:39.693847 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":251,"threads_started":1}
I20260808 13:14:39.740288 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:39.739537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:35085)
0808 13:14:39.740029 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:39.740115 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":271,"threads_started":1}
I20260808 13:14:40.230127 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:40.229310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:35049)
0808 13:14:40.229851 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:40.229941 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":274,"threads_started":1}
I20260808 13:14:40.695530 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:40.694830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:42107)
0808 13:14:40.695269 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:40.695348 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":214,"threads_started":1}
I20260808 13:14:40.742086 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:40.741276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:59623)
0808 13:14:40.741770 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:40.741875 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":230,"threads_started":1}
I20260808 13:14:41.231796 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:41.231035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:43055)
0808 13:14:41.231527 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:41.231615 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":247,"threads_started":1}
I20260808 13:14:41.526283 17754 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:41.526474 17754 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:41.527132 17754 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
W20260808 13:14:41.528012 17325 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
W20260808 13:14:41.528193 17324 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:41.528337 17324 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:41.528568 17754 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:14:41.697175 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:41.696453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:37697)
0808 13:14:41.696927 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:41.697014 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":232,"threads_started":1}
I20260808 13:14:41.743901 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:41.743067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:58479)
0808 13:14:41.743602 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:41.743689 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":260,"threads_started":1}
I20260808 13:14:42.233517 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:42.232632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:51443)
0808 13:14:42.233205 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:42.233306 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":309,"threads_started":1}
I20260808 13:14:42.698776 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:42.698028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:36263)
0808 13:14:42.698505 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:42.698592 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":251,"threads_started":1}
I20260808 13:14:42.745754 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:42.744881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:37903)
0808 13:14:42.745388 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:42.745516 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":251,"threads_started":1}
I20260808 13:14:43.235309 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:43.234475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:54615)
0808 13:14:43.235012 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:43.235100 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":243,"threads_started":1}
I20260808 13:14:43.700663 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:43.699802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:36085)
0808 13:14:43.700367 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:43.700462 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":238,"threads_started":1}
I20260808 13:14:43.747539 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:43.746774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:47021)
0808 13:14:43.747263 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:43.747356 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":239,"threads_started":1}
I20260808 13:14:43.839090 17763 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:43.839303 17763 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:43.840029 17763 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
W20260808 13:14:43.841104 17325 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
W20260808 13:14:43.841302 17324 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:43.841501 17324 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:43.841814 17763 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:14:44.237226 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:44.236372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:33291)
0808 13:14:44.236925 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:44.237031 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":293,"threads_started":1}
I20260808 13:14:44.702569 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:44.701651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:41879)
0808 13:14:44.702244 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:44.702347 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":334,"threads_started":1}
I20260808 13:14:44.749362 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:44.748556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:37571)
0808 13:14:44.749060 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:44.749157 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":240,"threads_started":1}
I20260808 13:14:45.239114 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:45.238243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:36123)
0808 13:14:45.238845 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:45.238954 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":302,"threads_started":1}
I20260808 13:14:45.704384 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:45.703464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:38363)
0808 13:14:45.704048 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:45.704161 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":308,"threads_started":1}
I20260808 13:14:45.751075 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:45.750333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:57329)
0808 13:14:45.750802 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:45.750904 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260808 13:14:46.240787 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:46.240052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:50527)
0808 13:14:46.240501 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:46.240595 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":240,"threads_started":1}
I20260808 13:14:46.706110 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:46.705295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:50489)
0808 13:14:46.705809 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:46.705902 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
I20260808 13:14:46.752866 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:46.752066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:53379)
0808 13:14:46.752585 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:46.752677 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
I20260808 13:14:47.242430 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:47.241657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:48145)
0808 13:14:47.242132 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:47.242237 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260808 13:14:47.707892 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:47.707023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:52185)
0808 13:14:47.707586 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:47.707701 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":295,"threads_started":1}
I20260808 13:14:47.754534 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:47.753753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:34147)
0808 13:14:47.754258 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:47.754361 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":260,"threads_started":1}
I20260808 13:14:48.243978 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:48.243281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:34937)
0808 13:14:48.243719 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:48.243801 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":223,"threads_started":1}
I20260808 13:14:48.709833 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:48.708998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:57365)
0808 13:14:48.709546 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:48.709639 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":265,"threads_started":1}
I20260808 13:14:48.756349 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:48.755479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:45693)
0808 13:14:48.756038 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:48.756144 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
I20260808 13:14:49.245781 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:49.244962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:58937)
0808 13:14:49.245480 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:49.245591 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":247,"threads_started":1}
I20260808 13:14:49.711568 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:49.710746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:48947)
0808 13:14:49.711289 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:49.711388 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":256,"threads_started":1}
I20260808 13:14:49.757951 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:49.757177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:49975)
0808 13:14:49.757669 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:49.757754 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":288,"threads_started":1}
I20260808 13:14:50.247395 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:50.246645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:52891)
0808 13:14:50.247097 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:50.247182 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260808 13:14:50.713198 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:50.712455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:51487)
0808 13:14:50.712903 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:50.712996 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260808 13:14:50.759964 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:50.759163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:37827)
0808 13:14:50.759679 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:50.759785 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":264,"threads_started":1}
I20260808 13:14:51.117374 17786 raft_consensus.cc:482] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:51.117623 17786 raft_consensus.cc:504] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8463024245f441629bee3e2615d8eb2d" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 39335 } } peers { permanent_uuid: "b1bdc9121ff14c55ac3ca01080a537ab" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34003 } } peers { permanent_uuid: "cd7e8b5e793b471cbd8aa9b6796187f6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 45387 } }
I20260808 13:14:51.118440 17786 leader_election.cc:288] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335), b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003)
W20260808 13:14:51.119462 17325 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8463024245f441629bee3e2615d8eb2d (127.31.59.131:39335): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
W20260808 13:14:51.119634 17324 leader_election.cc:341] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1bdc9121ff14c55ac3ca01080a537ab (127.31.59.130:34003): Not found: Tablet not found: 481228e64b51461aa4fd903d13f9c310
I20260808 13:14:51.119760 17324 leader_election.cc:302] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7e8b5e793b471cbd8aa9b6796187f6; no voters: 8463024245f441629bee3e2615d8eb2d, b1bdc9121ff14c55ac3ca01080a537ab
I20260808 13:14:51.119992 17786 raft_consensus.cc:2726] T 481228e64b51461aa4fd903d13f9c310 P cd7e8b5e793b471cbd8aa9b6796187f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:14:51.249505 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:51.248636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:48203)
0808 13:14:51.249147 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:51.249288 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":242,"threads_started":1}
I20260808 13:14:51.701399 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17262
I20260808 13:14:51.715206 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:51.714367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.131:50723)
0808 13:14:51.714885 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:51.715015 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":265,"threads_started":1}
I20260808 13:14:51.761853 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:51.760946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.129:38925)
0808 13:14:51.761489 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:51.761653 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":260,"threads_started":1}
I20260808 13:14:52.148855 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:51.702616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.31.59.129:45387)
0808 13:14:51.703135 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:51.703143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:52.130284 (+427141us) server_negotiation.cc:407] Connection header received
0808 13:14:52.130439 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:52.130447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:52.130711 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:52.130862 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:52.132374 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.133436 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.134791 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.135367 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.135469 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:52.143270 (+ 7801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:52.143287 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:52.143303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:52.143343 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:52.146363 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.147046 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.147053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.147056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.147185 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.147462 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.147464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.147466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.147861 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:52.148101 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:52.148318 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 13:14:52.148544 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260808 13:14:52.150218 17440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:52.242261 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17262
I20260808 13:14:52.251288 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:52.250451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:36101 (local address 127.31.59.130:34913)
0808 13:14:52.250993 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:52.251088 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.190:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":260,"threads_started":1}
I20260808 13:14:52.264817 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17448
I20260808 13:14:52.278355 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:52.265714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.31.59.130:34003)
0808 13:14:52.266318 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:52.266322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:52.266339 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:14:52.266396 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:52.266400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:52.266463 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:52.266551 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:52.267895 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.268685 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.269911 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.270335 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.270434 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:52.272970 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:52.272975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:52.272978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:52.273011 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:52.275644 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.276373 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.276377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.276380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.276518 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.276876 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.276879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.276881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.277134 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:52.277363 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:52.277819 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 13:14:52.277991 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":333,"threads_started":1}
I20260808 13:14:52.279058 17584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:52.364336 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17448
I20260808 13:14:52.385607 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17592
I20260808 13:14:52.399065 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:52.386471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.31.59.131:39335)
0808 13:14:52.387019 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:52.387024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:52.387042 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:14:52.387099 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:52.387103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:52.387165 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:52.387271 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:52.388598 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.389391 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.390704 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:52.391156 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:52.391258 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:52.393877 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:52.393882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:52.393885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:52.393917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:52.396512 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.397103 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.397106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.397108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.397240 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:52.397819 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:52.397822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:52.397824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:52.398092 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:52.398324 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:52.398526 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 13:14:52.398698 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":309,"threads_started":1}
I20260808 13:14:52.399766 17724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:52.485821 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17592
2026-08-08T13:14:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65356 ms)
[----------] 30 tests from SecurityITest (302554 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:14:52 dist-test-slave-49h3 krb5kdc[17799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:14:52 dist-test-slave-49h3 krb5kdc[17799](info): set up 2 sockets
Aug 08 13:14:52 dist-test-slave-49h3 krb5kdc[17799](info): commencing operation
krb5kdc: starting...
W20260808 13:14:53.582312 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:14:53 dist-test-slave-49h3 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:14:53.897053 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45619
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:54.242547 17812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:54.242841 17812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:54.242985 17812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:54.250294 17812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:14:54.250403 17812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:54.250475 17812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:54.250557 17812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:14:54.250619 17812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:14:54.260648 17812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:54.261987 17812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:54.263374 17812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:54.275450 17817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:54.275283 17818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:54.275485 17820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:54.276283 17812 server_base.cc:1034] running on GCE node
I20260808 13:14:54.277036 17812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:54.278470 17812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:54.279670 17812 hybrid_clock.cc:648] HybridClock initialized: now 1786194894279623 us; error 29 us; skew 500 ppm
I20260808 13:14:54.280007 17812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:54 dist-test-slave-49h3 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:54.283077 17812 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:14:54.284523 17812 webserver.cc:460] Webserver started at http://127.31.59.190:46385/ using document root <none> and password file <none>
I20260808 13:14:54.285202 17812 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:54.285319 17812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:54.285663 17812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:54.288637 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "b17e87010983496794c5eb74d61618f3"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "327c8b867cf72c6ef26438337ad5cd49"
server_key_iv: "9c01117379a4f9fdabc7595e9700c06c"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.289372 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "b17e87010983496794c5eb74d61618f3"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "327c8b867cf72c6ef26438337ad5cd49"
server_key_iv: "9c01117379a4f9fdabc7595e9700c06c"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.294378 17812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:14:54.297752 17827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:54.299041 17812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:14:54.299199 17812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "b17e87010983496794c5eb74d61618f3"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "327c8b867cf72c6ef26438337ad5cd49"
server_key_iv: "9c01117379a4f9fdabc7595e9700c06c"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.299377 17812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:54.328164 17812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:54.332396 17812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:54.332696 17812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:54.351112 17812 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:45619
I20260808 13:14:54.351114 17884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:45619 every 8 connection(s)
I20260808 13:14:54.352653 17812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:14:54.356415 17885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:54.356995 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17812
I20260808 13:14:54.357200 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:14:54.357908 31982 external_mini_cluster.cc:1428] Setting key 1856a1ac56dd0644d84e121950ffe763
I20260808 13:14:54.370007 17885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: Bootstrap starting.
Aug 08 13:14:54 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:54.373905 17885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:54.375001 17885 log.cc:824] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:54.378224 17885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: No bootstrap required, opened a new log
I20260808 13:14:54.381690 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:54.359882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.31.59.190:45619)
0808 13:14:54.360720 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:54.360732 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:54.360772 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:14:54.362483 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:54.362505 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:54.362887 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:54.363217 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:54.365006 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:54.366408 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:54.368311 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:54.368961 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:54.369089 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:54.374096 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:54.374111 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:54.374125 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:54.374167 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:54.377393 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:54.378227 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:54.378233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:54.378239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:54.378423 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:54.378867 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:54.378871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:54.378873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:54.379413 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:54.379632 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:54.380211 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 13:14:54.380635 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":344,"threads_started":1}
I20260808 13:14:54.386200 17885 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } }
I20260808 13:14:54.386543 17885 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:54.386637 17885 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17e87010983496794c5eb74d61618f3, State: Initialized, Role: FOLLOWER
I20260808 13:14:54.387140 17885 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } }
I20260808 13:14:54.387362 17885 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:14:54.387483 17885 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:14:54.387627 17885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:54.389973 17885 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } }
I20260808 13:14:54.390429 17885 leader_election.cc:302] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17e87010983496794c5eb74d61618f3; no voters:
I20260808 13:14:54.390982 17885 leader_election.cc:288] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:14:54.391103 17891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:14:54.391793 17891 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [term 1 LEADER]: Becoming Leader. State: Replica: b17e87010983496794c5eb74d61618f3, State: Running, Role: LEADER
I20260808 13:14:54.392288 17891 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } }
I20260808 13:14:54.392932 17885 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:14:54.394901 17893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17e87010983496794c5eb74d61618f3. Latest consensus state: current_term: 1 leader_uuid: "b17e87010983496794c5eb74d61618f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } } }
I20260808 13:14:54.395323 17893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:54.396396 17892 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17e87010983496794c5eb74d61618f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e87010983496794c5eb74d61618f3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45619 } } }
I20260808 13:14:54.396703 17892 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:54.402709 17898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:14:54.407183 17898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:14:54.415680 17898 catalog_manager.cc:1347] Generated new cluster ID: 93f415c3fb5442de92eced06e047a9db
I20260808 13:14:54.415787 17898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:14:54.436092 17898 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:14:54.437500 17898 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:14:54.444399 17898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: Generated new TSK 0
I20260808 13:14:54.445201 17898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:14:54.519165 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:54.863343 17912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:54.863646 17912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:54.863760 17912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:54.871393 17912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:54.871512 17912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:54.871636 17912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:14:54.881956 17912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:54.883388 17912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:54.884708 17912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:54.897686 17917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:54.897643 17918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:54.897723 17920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:54.898499 17912 server_base.cc:1034] running on GCE node
I20260808 13:14:54.899104 17912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:54.900077 17912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:54.901278 17912 hybrid_clock.cc:648] HybridClock initialized: now 1786194894901223 us; error 39 us; skew 500 ppm
I20260808 13:14:54.901618 17912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:54 dist-test-slave-49h3 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:54.905401 17912 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:14:54.907851 17912 webserver.cc:460] Webserver started at http://127.31.59.129:41161/ using document root <none> and password file <none>
I20260808 13:14:54.908512 17912 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:54.908619 17912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:54.908893 17912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:54.911804 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "5782b5ec9574485d8573ffef0f6b7272"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "e2285d23afba98f57728b9134d09193b"
server_key_iv: "dcc538f6dd15830a25276a6ff4ab1fd4"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.912573 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "5782b5ec9574485d8573ffef0f6b7272"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "e2285d23afba98f57728b9134d09193b"
server_key_iv: "dcc538f6dd15830a25276a6ff4ab1fd4"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.917572 17912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:14:54.920822 17927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:54.922083 17912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:14:54.922240 17912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "5782b5ec9574485d8573ffef0f6b7272"
format_stamp: "Formatted at 2026-08-08 13:14:54 on dist-test-slave-49h3"
server_key: "e2285d23afba98f57728b9134d09193b"
server_key_iv: "dcc538f6dd15830a25276a6ff4ab1fd4"
server_key_version: "encryptionkey@0"
I20260808 13:14:54.922435 17912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:54.948736 17912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:54.952965 17912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:54.953245 17912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:54.954263 17912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:54.955593 17912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:54.955670 17912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:54.955802 17912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:54.955854 17912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:54.987637 17912 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:41537
I20260808 13:14:54.987713 18046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:41537 every 8 connection(s)
I20260808 13:14:54.988948 17912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:14:54.990718 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 17912
I20260808 13:14:54.990967 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:14:54.991657 31982 external_mini_cluster.cc:1428] Setting key c80277098590b2df5d02933967233311
Aug 08 13:14:54 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:55.009884 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:54.991788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:35347 (local address 127.31.59.190:45619)
0808 13:14:54.992327 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:54.992334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:54.995429 (+ 3095us) server_negotiation.cc:407] Connection header received
0808 13:14:54.995487 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:54.995491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:54.995556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:54.995647 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:54.997283 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:54.998099 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:54.999756 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.000177 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.000307 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:55.004018 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:55.004024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:55.004027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:55.004062 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:55.006641 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:55.007541 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:55.007545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:55.007548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:55.007682 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:55.008147 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:55.008150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:55.008152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:55.008420 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:55.008533 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:55.009357 (+ 824us) server_negotiation.cc:299] Negotiation successful
0808 13:14:55.009606 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260808 13:14:55.011236 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:54.992341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45619 (local address 127.31.59.129:35347)
0808 13:14:54.993262 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:54.993310 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:54.995158 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:54.995921 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:54.995930 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:54.996518 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:54.997098 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:54.997116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:54.998283 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:54.998287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:54.999599 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:54.999617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.000435 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.000443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:55.000672 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:55.001320 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:55.001345 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:55.003812 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:55.006780 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:55.006790 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:55.006808 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:55.007369 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:55.007788 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:55.007793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:55.007796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:55.008049 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:55.008544 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:55.008552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:55.008974 (+ 422us) client_negotiation.cc:769] Sending connection context
0808 13:14:55.009274 (+ 300us) client_negotiation.cc:240] Negotiation successful
0808 13:14:55.009849 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":387,"threads_started":1}
I20260808 13:14:55.012778 18047 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45619
I20260808 13:14:55.013113 18047 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:55.014051 18047 heartbeater.cc:511] Master 127.31.59.190:45619 requested a full tablet report, sending...
I20260808 13:14:55.016201 17846 ts_manager.cc:194] Registered new tserver with Master: 5782b5ec9574485d8573ffef0f6b7272 (127.31.59.129:41537)
I20260808 13:14:55.017719 17846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:35347
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:14:55.060833 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:55.406392 18056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:55.406689 18056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:55.406836 18056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:55.414505 18056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:55.414651 18056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:55.414821 18056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:14:55.424667 18056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:55.426168 18056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:55.427472 18056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:55.440493 18062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:55.440605 18061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:55.440635 18064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:55.441666 18056 server_base.cc:1034] running on GCE node
I20260808 13:14:55.442281 18056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:55.443194 18056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:55.444401 18056 hybrid_clock.cc:648] HybridClock initialized: now 1786194895444347 us; error 36 us; skew 500 ppm
I20260808 13:14:55.444757 18056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:55 dist-test-slave-49h3 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194895, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:55.448030 18056 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:14:55.449563 18056 webserver.cc:460] Webserver started at http://127.31.59.130:36541/ using document root <none> and password file <none>
I20260808 13:14:55.450222 18056 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:55.450353 18056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:55.450659 18056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:55.453598 18056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "98e625e3a232409fba4bb70fdd9b0a35"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "b70986750870982b618ad1aefd25fce5"
server_key_iv: "5966b83df04ca591fb66af10d7dd2f3b"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.454356 18056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "98e625e3a232409fba4bb70fdd9b0a35"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "b70986750870982b618ad1aefd25fce5"
server_key_iv: "5966b83df04ca591fb66af10d7dd2f3b"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.459357 18056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:14:55.462692 18071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:55.463936 18056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:14:55.464085 18056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "98e625e3a232409fba4bb70fdd9b0a35"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "b70986750870982b618ad1aefd25fce5"
server_key_iv: "5966b83df04ca591fb66af10d7dd2f3b"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.464273 18056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:55.482725 18056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:55.487012 18056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:55.487317 18056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:55.488279 18056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:55.489650 18056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:55.489730 18056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:55.489868 18056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:55.489923 18056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:55.521905 18056 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:33339
I20260808 13:14:55.521965 18190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:33339 every 8 connection(s)
I20260808 13:14:55.523200 18056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:14:55.529302 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18056
I20260808 13:14:55.529536 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:14:55.530133 31982 external_mini_cluster.cc:1428] Setting key 9d23ac5f225ab2014ba0fb84d70fd6cf
Aug 08 13:14:55 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194895, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:55.544697 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:55.525693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:42151 (local address 127.31.59.190:45619)
0808 13:14:55.526223 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:55.526230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:55.527243 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 13:14:55.529129 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:55.529133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:55.529207 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:55.529303 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:55.531375 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.532438 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.534641 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.535080 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.535169 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:55.539193 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:55.539198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:55.539201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:55.539234 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:55.541761 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:55.542563 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:55.542567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:55.542569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:55.542696 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:55.543123 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:55.543126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:55.543128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:55.543375 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:55.543480 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:55.544246 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 13:14:55.544412 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":258,"threads_started":1}
I20260808 13:14:55.546088 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:55.526308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45619 (local address 127.31.59.130:42151)
0808 13:14:55.527085 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:55.527129 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:55.528824 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:55.529620 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:55.529632 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:55.530421 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:55.531173 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:55.531198 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.532600 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.532605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:55.534501 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:55.534518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:55.535318 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:55.535325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:55.535600 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:55.536286 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:55.536315 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:55.538954 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:55.541891 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:55.541900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:55.541916 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:55.542470 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:55.542799 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:55.542803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:55.542806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:55.543035 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:55.543485 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:55.543492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:55.543897 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 13:14:55.544189 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 13:14:55.544708 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":279,"threads_started":1}
I20260808 13:14:55.547616 18191 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45619
I20260808 13:14:55.547904 18191 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:55.548615 18191 heartbeater.cc:511] Master 127.31.59.190:45619 requested a full tablet report, sending...
I20260808 13:14:55.550143 17846 ts_manager.cc:194] Registered new tserver with Master: 98e625e3a232409fba4bb70fdd9b0a35 (127.31.59.130:33339)
I20260808 13:14:55.551210 17846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:42151
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:14:55.597532 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:14:55.933115 18198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:55.933414 18198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:55.933570 18198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:55.941795 18198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:55.941951 18198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:55.942128 18198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:14:55.952663 18198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:45619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:55.954020 18198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:55.955399 18198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:55.968315 18204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:55.968357 18207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:55.968380 18203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:55.969051 18198 server_base.cc:1034] running on GCE node
I20260808 13:14:55.969748 18198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:55.970692 18198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:55.971923 18198 hybrid_clock.cc:648] HybridClock initialized: now 1786194895971861 us; error 49 us; skew 500 ppm
I20260808 13:14:55.972262 18198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:14:55 dist-test-slave-49h3 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194895, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:14:55.975416 18198 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:14:55.976890 18198 webserver.cc:460] Webserver started at http://127.31.59.131:36135/ using document root <none> and password file <none>
I20260808 13:14:55.977608 18198 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:55.977728 18198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:55.978031 18198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:55.981010 18198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "f5344158910841579dd880ee7cacb725"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "92d2f5987e031bda1ed534c01a3a6beb"
server_key_iv: "b9d837a4452166bd2b6920dd24614d3a"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.981812 18198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "f5344158910841579dd880ee7cacb725"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "92d2f5987e031bda1ed534c01a3a6beb"
server_key_iv: "b9d837a4452166bd2b6920dd24614d3a"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.987092 18198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:14:55.990569 18216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:55.991963 18198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:14:55.992106 18198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "f5344158910841579dd880ee7cacb725"
format_stamp: "Formatted at 2026-08-08 13:14:55 on dist-test-slave-49h3"
server_key: "92d2f5987e031bda1ed534c01a3a6beb"
server_key_iv: "b9d837a4452166bd2b6920dd24614d3a"
server_key_version: "encryptionkey@0"
I20260808 13:14:55.992304 18198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:56.013639 18198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:56.017988 18198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:56.018267 18198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:56.019265 18198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:56.020583 18198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:56.020661 18198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:56.020776 18198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:56.020826 18198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:56.021284 18047 heartbeater.cc:503] Master 127.31.59.190:45619 was elected leader, sending a full tablet report...
I20260808 13:14:56.052220 18198 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:38359
I20260808 13:14:56.052279 18335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:38359 every 8 connection(s)
I20260808 13:14:56.053546 18198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:14:56.057410 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18198
I20260808 13:14:56.057641 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:14:56.058224 31982 external_mini_cluster.cc:1428] Setting key b8f8dfb2542931f034ff1eea301041c1
Aug 08 13:14:56 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194895, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:14:56.073225 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.056032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:43573 (local address 127.31.59.190:45619)
0808 13:14:56.056508 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.056514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.057632 (+ 1118us) server_negotiation.cc:407] Connection header received
0808 13:14:56.059750 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.059754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.059839 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.059925 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:56.061440 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.062222 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.063657 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.064040 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.064129 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.067463 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:56.067471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:56.067476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:56.067516 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:56.070148 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.070821 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.070825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.070827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.070949 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.071353 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.071358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.071360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.071646 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:56.071793 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.072555 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.072800 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":201,"threads_started":1}
I20260808 13:14:56.074150 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.056598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:45619 (local address 127.31.59.131:43573)
0808 13:14:56.057418 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.057508 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.059505 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.060170 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.060178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.060698 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:56.061267 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.061298 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.062346 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.062350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.063550 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.063562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.064202 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.064206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.064405 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.065019 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:14:56.065042 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:56.067213 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:56.070282 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:56.070291 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:56.070307 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:56.070709 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:56.071050 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:14:56.071054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:14:56.071058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:14:56.071248 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:14:56.071765 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:56.071773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:56.072172 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.072474 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.072986 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":288,"threads_started":1}
I20260808 13:14:56.075331 18336 heartbeater.cc:347] Connected to a master server at 127.31.59.190:45619
I20260808 13:14:56.075619 18336 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:56.076315 18336 heartbeater.cc:511] Master 127.31.59.190:45619 requested a full tablet report, sending...
I20260808 13:14:56.077683 17846 ts_manager.cc:194] Registered new tserver with Master: f5344158910841579dd880ee7cacb725 (127.31.59.131:38359)
I20260808 13:14:56.078454 17846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:43573
I20260808 13:14:56.085608 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:14:56.101899 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.090071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.31.59.190:45619)
0808 13:14:56.090273 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.090279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.090658 (+ 379us) server_negotiation.cc:407] Connection header received
0808 13:14:56.090782 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.090786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.090865 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.090941 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:56.092284 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.093016 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.094314 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.094714 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.094806 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.097139 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:56.097143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:56.097146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:56.097176 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:56.099704 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.100284 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.100287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.100289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.100407 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.100679 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.100682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.100684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.100976 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:56.101150 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.101440 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.101637 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:14:56.105623 17846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:14:56.112303 17846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:14:56.147732 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.124367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.31.59.131:38359)
0808 13:14:56.139236 (+ 14869us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.139245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.139288 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:14:56.139394 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.139402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.139667 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.139859 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:56.140613 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.143111 (+ 2498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.144933 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.146335 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.146462 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.146711 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.147220 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.147399 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14685,"thread_start_us":14521,"threads_started":1}
I20260808 13:14:56.147794 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.124082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:38359 (local address 127.0.0.1:41772)
0808 13:14:56.138226 (+ 14144us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.138296 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.138524 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.139897 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.139904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.139935 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:56.140384 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.140395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.143248 (+ 2853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.143254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.144756 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.144769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.146852 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.146856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.146992 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.147033 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.147142 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.147360 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13873,"mutex_wait_us":34,"thread_start_us":15023,"threads_started":1}
I20260808 13:14:56.148150 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.124798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.31.59.130:33339)
0808 13:14:56.137315 (+ 12517us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.137325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.139186 (+ 1861us) server_negotiation.cc:407] Connection header received
0808 13:14:56.139514 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.139523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.139776 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.139946 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:56.141199 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.142517 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.145271 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.146748 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.146892 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.147071 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.147634 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.147819 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12258,"thread_start_us":12531,"threads_started":1}
I20260808 13:14:56.148562 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.124430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:33339 (local address 127.0.0.1:54196)
0808 13:14:56.139039 (+ 14609us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.139063 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.139303 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.140245 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.140251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.140282 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:56.140988 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.141001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.142690 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.142695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.145103 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.145119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.147223 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.147230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.147413 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.147447 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.148087 (+ 640us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.148184 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14477,"mutex_wait_us":1,"thread_start_us":13349,"threads_started":1}
I20260808 13:14:56.149240 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.140304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46032)
0808 13:14:56.141484 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.141500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.141626 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.142300 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.142305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.142330 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:56.142765 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.142776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.144726 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.144730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.146618 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.146632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.148436 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.148442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.148691 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.148724 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.148817 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.148936 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"thread_start_us":849,"threads_started":1}
I20260808 13:14:56.149538 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.139914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.31.59.129:41537)
0808 13:14:56.140967 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.140975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.141609 (+ 634us) server_negotiation.cc:407] Connection header received
0808 13:14:56.141833 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.141841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.142096 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.142304 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:56.142983 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.144531 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.146813 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.148228 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.148374 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.148587 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.149033 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.149193 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":565,"threads_started":1}
I20260808 13:14:56.152127 18121 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 (DEFAULT_TABLE table=test-table [id=df0b9ea9199e40298142ff5b46943888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:56.152128 18266 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 (DEFAULT_TABLE table=test-table [id=df0b9ea9199e40298142ff5b46943888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:56.153363 18266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab3365cb70443dbaef8bc6124ae2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:56.153437 18121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab3365cb70443dbaef8bc6124ae2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:56.156258 17977 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 (DEFAULT_TABLE table=test-table [id=df0b9ea9199e40298142ff5b46943888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:56.157732 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab3365cb70443dbaef8bc6124ae2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:56.166285 18356 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Bootstrap starting.
I20260808 13:14:56.168380 18357 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Bootstrap starting.
I20260808 13:14:56.171126 18356 tablet_bootstrap.cc:654] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:56.172302 18357 tablet_bootstrap.cc:654] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:56.172467 18356 log.cc:824] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:56.173112 18358 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Bootstrap starting.
I20260808 13:14:56.173606 18357 log.cc:824] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:56.176203 18356 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: No bootstrap required, opened a new log
I20260808 13:14:56.176466 18356 ts_tablet_manager.cc:1397] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 13:14:56.176586 18357 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: No bootstrap required, opened a new log
I20260808 13:14:56.176681 18358 tablet_bootstrap.cc:654] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:56.176864 18357 ts_tablet_manager.cc:1397] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 13:14:56.177953 18358 log.cc:824] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:56.182348 18358 tablet_bootstrap.cc:492] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: No bootstrap required, opened a new log
I20260808 13:14:56.182626 18358 ts_tablet_manager.cc:1397] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 13:14:56.184814 18356 raft_consensus.cc:348] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.185194 18356 raft_consensus.cc:374] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:56.185307 18356 raft_consensus.cc:729] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e625e3a232409fba4bb70fdd9b0a35, State: Initialized, Role: FOLLOWER
I20260808 13:14:56.185904 18356 consensus_queue.cc:260] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.187121 18191 heartbeater.cc:503] Master 127.31.59.190:45619 was elected leader, sending a full tablet report...
I20260808 13:14:56.186961 18357 raft_consensus.cc:348] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.187371 18357 raft_consensus.cc:374] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:56.187461 18357 raft_consensus.cc:729] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5344158910841579dd880ee7cacb725, State: Initialized, Role: FOLLOWER
I20260808 13:14:56.187723 18356 ts_tablet_manager.cc:1428] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 13:14:56.188189 18357 consensus_queue.cc:260] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.189286 18336 heartbeater.cc:503] Master 127.31.59.190:45619 was elected leader, sending a full tablet report...
I20260808 13:14:56.189881 18357 ts_tablet_manager.cc:1428] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260808 13:14:56.192655 18358 raft_consensus.cc:348] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.193162 18358 raft_consensus.cc:374] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:56.193267 18358 raft_consensus.cc:729] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5782b5ec9574485d8573ffef0f6b7272, State: Initialized, Role: FOLLOWER
I20260808 13:14:56.193853 18358 consensus_queue.cc:260] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.195276 18358 ts_tablet_manager.cc:1428] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260808 13:14:56.241530 18048 tablet.cc:2369] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:14:56.263403 18365 raft_consensus.cc:482] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:56.263805 18365 raft_consensus.cc:504] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.265861 18365 leader_election.cc:288] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5344158910841579dd880ee7cacb725 (127.31.59.131:38359), 98e625e3a232409fba4bb70fdd9b0a35 (127.31.59.130:33339)
I20260808 13:14:56.272578 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.266364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:45803 (local address 127.31.59.130:33339)
0808 13:14:56.266572 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.266578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.267240 (+ 662us) server_negotiation.cc:407] Connection header received
0808 13:14:56.267401 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.267405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.267467 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.267537 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:56.268240 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.268974 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.270890 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.271650 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.271782 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.271855 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.272201 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.272329 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:14:56.272603 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.266168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:43345 (local address 127.31.59.131:38359)
0808 13:14:56.266401 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.266408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.267158 (+ 750us) server_negotiation.cc:407] Connection header received
0808 13:14:56.267399 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.267404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.267466 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.267548 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:56.268231 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.269012 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.270895 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.271687 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.271849 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.271923 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.272221 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.272359 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:14:56.272847 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.266067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:38359 (local address 127.31.59.129:43345)
0808 13:14:56.267045 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.267060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.267208 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.267580 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.267584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.267608 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:56.268080 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.268094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.269156 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.269164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.270743 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.270755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.271922 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.271929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.272068 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.272090 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.272173 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.272308 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":54,"thread_start_us":651,"threads_started":1}
I20260808 13:14:56.272964 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.266343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:33339 (local address 127.31.59.129:45803)
0808 13:14:56.267157 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.267171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:56.267279 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:56.267591 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:56.267596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:56.267619 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:56.268090 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.268100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.269097 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.269102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.270715 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:56.270731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.271873 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.271880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:56.272038 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.272069 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:14:56.272143 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 13:14:56.272269 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":24,"thread_start_us":459,"threads_started":1}
I20260808 13:14:56.273878 18142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab3365cb70443dbaef8bc6124ae2a1" candidate_uuid: "5782b5ec9574485d8573ffef0f6b7272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e625e3a232409fba4bb70fdd9b0a35" is_pre_election: true
I20260808 13:14:56.273881 18288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab3365cb70443dbaef8bc6124ae2a1" candidate_uuid: "5782b5ec9574485d8573ffef0f6b7272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5344158910841579dd880ee7cacb725" is_pre_election: true
I20260808 13:14:56.274360 18142 raft_consensus.cc:2454] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5782b5ec9574485d8573ffef0f6b7272 in term 0.
I20260808 13:14:56.274360 18288 raft_consensus.cc:2454] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5782b5ec9574485d8573ffef0f6b7272 in term 0.
I20260808 13:14:56.275040 17928 leader_election.cc:302] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5782b5ec9574485d8573ffef0f6b7272, 98e625e3a232409fba4bb70fdd9b0a35; no voters:
W20260808 13:14:56.275233 18192 tablet.cc:2369] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:14:56.275501 18365 raft_consensus.cc:2781] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:14:56.275635 18365 raft_consensus.cc:482] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:14:56.275756 18365 raft_consensus.cc:3037] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:56.278009 18365 raft_consensus.cc:504] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.278646 18365 leader_election.cc:288] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [CANDIDATE]: Term 1 election: Requested vote from peers f5344158910841579dd880ee7cacb725 (127.31.59.131:38359), 98e625e3a232409fba4bb70fdd9b0a35 (127.31.59.130:33339)
I20260808 13:14:56.279100 18288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab3365cb70443dbaef8bc6124ae2a1" candidate_uuid: "5782b5ec9574485d8573ffef0f6b7272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5344158910841579dd880ee7cacb725"
I20260808 13:14:56.279152 18142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab3365cb70443dbaef8bc6124ae2a1" candidate_uuid: "5782b5ec9574485d8573ffef0f6b7272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e625e3a232409fba4bb70fdd9b0a35"
I20260808 13:14:56.279335 18288 raft_consensus.cc:3037] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:56.279381 18142 raft_consensus.cc:3037] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:56.281589 18288 raft_consensus.cc:2454] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5782b5ec9574485d8573ffef0f6b7272 in term 1.
I20260808 13:14:56.281636 18142 raft_consensus.cc:2454] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5782b5ec9574485d8573ffef0f6b7272 in term 1.
I20260808 13:14:56.281998 17930 leader_election.cc:302] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5782b5ec9574485d8573ffef0f6b7272, f5344158910841579dd880ee7cacb725; no voters:
I20260808 13:14:56.282313 18365 raft_consensus.cc:2781] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:14:56.282747 18365 raft_consensus.cc:686] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 1 LEADER]: Becoming Leader. State: Replica: 5782b5ec9574485d8573ffef0f6b7272, State: Running, Role: LEADER
I20260808 13:14:56.283246 18365 consensus_queue.cc:237] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } }
I20260808 13:14:56.287707 17844 catalog_manager.cc:5495] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5782b5ec9574485d8573ffef0f6b7272 (127.31.59.129). New cstate: current_term: 1 leader_uuid: "5782b5ec9574485d8573ffef0f6b7272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5782b5ec9574485d8573ffef0f6b7272" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41537 } health_report { overall_health: HEALTHY } } }
W20260808 13:14:56.305119 18337 tablet.cc:2369] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:14:56.306919 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.301325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.31.59.129:41537)
0808 13:14:56.301557 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.301563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.301584 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:14:56.301740 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.301744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.301880 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.302022 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:14:56.302419 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.303182 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.304640 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.305255 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.305377 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.305943 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:14:56.306485 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:14:56.306594 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.306647 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.306713 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:14:56.314754 18288 raft_consensus.cc:1261] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 1 FOLLOWER]: Refusing update from remote peer 5782b5ec9574485d8573ffef0f6b7272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:56.314769 18142 raft_consensus.cc:1261] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 1 FOLLOWER]: Refusing update from remote peer 5782b5ec9574485d8573ffef0f6b7272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:56.315696 18368 consensus_queue.cc:1035] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e625e3a232409fba4bb70fdd9b0a35" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:56.316078 18365 consensus_queue.cc:1035] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5344158910841579dd880ee7cacb725" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:56.327235 18372 mvcc.cc:205] Tried to move back new op lower bound from 7316254295295668224 to 7316254295179747328. Current Snapshot: MvccSnapshot[applied={T|T < 7316254295295668224}]
I20260808 13:14:56.328035 18371 mvcc.cc:205] Tried to move back new op lower bound from 7316254295295668224 to 7316254295179747328. Current Snapshot: MvccSnapshot[applied={T|T < 7316254295295668224}]
I20260808 13:14:56.339696 17844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:56.339963 17844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:56.343616 17844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b17e87010983496794c5eb74d61618f3: Sending DeleteTablet for 3 replicas of tablet e9ab3365cb70443dbaef8bc6124ae2a1
I20260808 13:14:56.344934 18121 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:56 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:54196
I20260808 13:14:56.344934 18266 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:56 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:41772
I20260808 13:14:56.345063 17977 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ab3365cb70443dbaef8bc6124ae2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:56 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46032
I20260808 13:14:56.347440 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17912
I20260808 13:14:56.374333 18382 tablet_replica.cc:330] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: stopping tablet replica
I20260808 13:14:56.375031 18382 raft_consensus.cc:2229] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:56.375716 18382 raft_consensus.cc:2258] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:56.380146 18382 ts_tablet_manager.cc:1905] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 13:14:56 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:14:56.383420 18381 tablet_replica.cc:330] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: stopping tablet replica
I20260808 13:14:56.383693 18383 tablet_replica.cc:330] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: stopping tablet replica
I20260808 13:14:56.384115 18381 raft_consensus.cc:2229] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:56.384114 18383 raft_consensus.cc:2229] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:14:56.384722 18381 raft_consensus.cc:2258] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:56.384806 18383 raft_consensus.cc:2258] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:56.387259 18381 ts_tablet_manager.cc:1905] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:56.394116 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.349166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.31.59.129:41537)
0808 13:14:56.349383 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.349390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.370785 (+ 21395us) server_negotiation.cc:407] Connection header received
0808 13:14:56.371338 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.371345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.371441 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.371507 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:56.373323 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.374555 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.377310 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.378016 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.378125 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.386412 (+ 8287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:56.386428 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:56.386445 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:56.386501 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:56.390520 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.391471 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.391479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.391483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.391661 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.392134 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.392139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.392142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.392594 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:56.392887 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.393497 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.393758 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:14:56.394335 18382 ts_tablet_manager.cc:1918] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:56.394510 18382 log.cc:1196] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/e9ab3365cb70443dbaef8bc6124ae2a1
I20260808 13:14:56.394752 18383 ts_tablet_manager.cc:1905] T e9ab3365cb70443dbaef8bc6124ae2a1 P 5782b5ec9574485d8573ffef0f6b7272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:56.395051 18382 ts_tablet_manager.cc:1939] T e9ab3365cb70443dbaef8bc6124ae2a1 P f5344158910841579dd880ee7cacb725: Deleting consensus metadata
I20260808 13:14:56.396119 18042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:56.396991 17830 catalog_manager.cc:4841] TS f5344158910841579dd880ee7cacb725 (127.31.59.131:38359): tablet e9ab3365cb70443dbaef8bc6124ae2a1 (table test-table [id=df0b9ea9199e40298142ff5b46943888]) successfully deleted
I20260808 13:14:56.398000 18381 ts_tablet_manager.cc:1918] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:56.398157 18381 log.cc:1196] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/e9ab3365cb70443dbaef8bc6124ae2a1
I20260808 13:14:56.398720 18381 ts_tablet_manager.cc:1939] T e9ab3365cb70443dbaef8bc6124ae2a1 P 98e625e3a232409fba4bb70fdd9b0a35: Deleting consensus metadata
I20260808 13:14:56.400359 17828 catalog_manager.cc:4841] TS 98e625e3a232409fba4bb70fdd9b0a35 (127.31.59.130:33339): tablet e9ab3365cb70443dbaef8bc6124ae2a1 (table test-table [id=df0b9ea9199e40298142ff5b46943888]) successfully deleted
I20260808 13:14:56.497871 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17912
I20260808 13:14:56.518603 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18056
W20260808 13:14:56.518560 17830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:41537 (error 108)
I20260808 13:14:56.520426 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.519968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46058)
0808 13:14:56.520149 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.520282 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 13:14:56.520692 17830 catalog_manager.cc:4584] TS 5782b5ec9574485d8573ffef0f6b7272 (127.31.59.129:41537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9ab3365cb70443dbaef8bc6124ae2a1: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Aug 08 13:14:56 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:14:56.536424 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.519589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.31.59.130:33339)
0808 13:14:56.519801 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.519807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.519825 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:14:56.519879 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.519883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.519952 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.520035 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:56.522312 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.523108 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.524993 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.525405 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.525559 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.530606 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:56.530619 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:56.530632 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:56.530669 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:56.533671 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.534334 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.534341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.534343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.534472 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.534839 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.534843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.534845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.535266 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:56.535553 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.535870 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.536110 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:14:56.537052 18185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:56.574458 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.573967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46062)
0808 13:14:56.574173 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.574294 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:14:56.575603 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.575199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46068)
0808 13:14:56.575348 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.575439 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 13:14:56.628386 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18056
I20260808 13:14:56.636279 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.635699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46074)
0808 13:14:56.635944 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.636059 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 13:14:56.637383 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.636986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46082)
0808 13:14:56.637137 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.637233 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 13:14:56.647819 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18198
Aug 08 13:14:56 dist-test-slave-49h3 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:14:56.662860 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.648597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.31.59.131:38359)
0808 13:14:56.648794 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:56.648799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:56.648819 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:14:56.648969 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:56.648973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:56.649071 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:56.649156 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:56.650494 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.651231 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.652575 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:56.652985 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:56.653092 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:56.657375 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:56.657391 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:56.657407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:14:56.657440 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:56.660108 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.660782 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.660789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.660791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.660939 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:14:56.661305 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:14:56.661309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:14:56.661311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:14:56.661712 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:56.662018 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:56.662305 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:14:56.662534 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:14:56.663496 18331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:56.717993 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.717439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46084)
0808 13:14:56.717713 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.717823 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:14:56.719009 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:56.718621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41537 (local address 127.0.0.1:46086)
0808 13:14:56.718824 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:56.718898 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 13:14:56.753959 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18198
I20260808 13:14:56.771533 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 17812
I20260808 13:14:56.772912 17880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:56.885572 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 17812
2026-08-08T13:14:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4402 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T13:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:14:56Z Disabled control of system clock
I20260808 13:14:56.978684 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40365
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:14:57.316144 18394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:57.316434 18394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:57.316527 18394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:57.323764 18394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:14:57.323875 18394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:57.323920 18394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:57.323977 18394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:14:57.324021 18394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:14:57.333479 18394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:57.334769 18394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:57.336520 18394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:57.347838 18400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:57.347883 18404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:57.347913 18401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:57.348718 18394 server_base.cc:1034] running on GCE node
I20260808 13:14:57.349392 18394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:57.350745 18394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:57.351948 18394 hybrid_clock.cc:648] HybridClock initialized: now 1786194897351889 us; error 50 us; skew 500 ppm
I20260808 13:14:57.352286 18394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:14:57.357745 18394 webserver.cc:460] Webserver started at http://127.31.59.190:40113/ using document root <none> and password file <none>
I20260808 13:14:57.358415 18394 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:57.358521 18394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:57.358842 18394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:57.362023 18394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "442210998529495788d92b1a9b978dae"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "4759ed355935849014e023b5d712c705"
server_key_iv: "5b34c131ca44153664a0636c2cd0832f"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.362876 18394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "442210998529495788d92b1a9b978dae"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "4759ed355935849014e023b5d712c705"
server_key_iv: "5b34c131ca44153664a0636c2cd0832f"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.367959 18394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:14:57.371268 18410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:57.372521 18394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:14:57.372684 18394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "442210998529495788d92b1a9b978dae"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "4759ed355935849014e023b5d712c705"
server_key_iv: "5b34c131ca44153664a0636c2cd0832f"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.372884 18394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:57.386543 18394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:57.387327 18394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:57.387610 18394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:57.405486 18394 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40365
I20260808 13:14:57.405511 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40365 every 8 connection(s)
I20260808 13:14:57.406824 18394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:14:57.408535 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18394
I20260808 13:14:57.408702 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:14:57.409142 31982 external_mini_cluster.cc:1428] Setting key 6d73c71f731faeba3eca099ffd38ed2f
I20260808 13:14:57.410569 18464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:57.422103 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:57.411065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.31.59.190:40365)
0808 13:14:57.412026 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:57.412037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:57.412077 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:14:57.413715 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:57.413735 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:57.414169 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:57.414535 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:57.415151 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.416468 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.417879 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.418544 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.418721 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:57.419891 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:57.419898 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:57.419913 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:57.420461 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:57.420542 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:57.420563 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:57.420666 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:57.420880 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 13:14:57.421156 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":361,"threads_started":1}
I20260808 13:14:57.422348 18464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: Bootstrap starting.
I20260808 13:14:57.425595 18464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:57.426528 18464 log.cc:824] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:57.429352 18464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: No bootstrap required, opened a new log
I20260808 13:14:57.436573 18464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } }
I20260808 13:14:57.436888 18464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:57.436964 18464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442210998529495788d92b1a9b978dae, State: Initialized, Role: FOLLOWER
I20260808 13:14:57.437462 18464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } }
I20260808 13:14:57.437692 18464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:14:57.437791 18464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:14:57.437911 18464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:57.439939 18464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } }
I20260808 13:14:57.440359 18464 leader_election.cc:302] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 442210998529495788d92b1a9b978dae; no voters:
I20260808 13:14:57.440985 18464 leader_election.cc:288] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:14:57.441169 18470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:14:57.442040 18470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [term 1 LEADER]: Becoming Leader. State: Replica: 442210998529495788d92b1a9b978dae, State: Running, Role: LEADER
I20260808 13:14:57.442597 18470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } }
I20260808 13:14:57.443338 18464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:14:57.445406 18473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 442210998529495788d92b1a9b978dae. Latest consensus state: current_term: 1 leader_uuid: "442210998529495788d92b1a9b978dae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } } }
I20260808 13:14:57.445875 18473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:57.445662 18472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "442210998529495788d92b1a9b978dae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442210998529495788d92b1a9b978dae" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40365 } } }
I20260808 13:14:57.446244 18472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae [sys.catalog]: This master's current role is: LEADER
I20260808 13:14:57.447489 18479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:14:57.452276 18479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:14:57.460348 18479 catalog_manager.cc:1347] Generated new cluster ID: 5e456017844a4dc3b07eaa83989ddc02
I20260808 13:14:57.460474 18479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:14:57.481336 18479 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:14:57.483139 18479 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:14:57.495093 18479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: Generated new TSK 0
I20260808 13:14:57.495970 18479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:14:57.511933 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--builtin_ntp_servers=127.31.59.148:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:14:57.857380 18492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:57.857738 18492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:57.857899 18492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:57.865435 18492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:57.865594 18492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:57.865767 18492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:14:57.875910 18492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:57.877326 18492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:57.879388 18492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:57.893055 18498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:57.893165 18497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:57.893349 18501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:57.894126 18492 server_base.cc:1034] running on GCE node
I20260808 13:14:57.894739 18492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:57.895700 18492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:57.896893 18492 hybrid_clock.cc:648] HybridClock initialized: now 1786194897896836 us; error 50 us; skew 500 ppm
I20260808 13:14:57.897207 18492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:14:57.899502 18492 webserver.cc:460] Webserver started at http://127.31.59.129:35815/ using document root <none> and password file <none>
I20260808 13:14:57.900198 18492 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:57.900309 18492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:57.900620 18492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:57.903705 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "4b3c2c3595d046ff85e3e4945f695a4c"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "b2ed18427aa44da4cfb06d5090a388fc"
server_key_iv: "465001992ad06c6532ed7abb99b59687"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.904531 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "4b3c2c3595d046ff85e3e4945f695a4c"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "b2ed18427aa44da4cfb06d5090a388fc"
server_key_iv: "465001992ad06c6532ed7abb99b59687"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.909881 18492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:14:57.913646 18508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:57.915311 18492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:14:57.915496 18492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "4b3c2c3595d046ff85e3e4945f695a4c"
format_stamp: "Formatted at 2026-08-08 13:14:57 on dist-test-slave-49h3"
server_key: "b2ed18427aa44da4cfb06d5090a388fc"
server_key_iv: "465001992ad06c6532ed7abb99b59687"
server_key_version: "encryptionkey@0"
I20260808 13:14:57.915745 18492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:57.932689 18492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:57.933580 18492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:57.933866 18492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:57.934834 18492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:57.938524 18492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:57.938642 18492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:57.938756 18492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:57.938827 18492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:57.974169 18492 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:36379
I20260808 13:14:57.974270 18633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:36379 every 8 connection(s)
I20260808 13:14:57.975549 18492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:14:57.984705 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18492
I20260808 13:14:57.984896 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:14:57.985371 31982 external_mini_cluster.cc:1428] Setting key 98c73268508e678ee59a477aba89a2d6
I20260808 13:14:57.988615 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:57.978142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:42269 (local address 127.31.59.190:40365)
0808 13:14:57.978605 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:57.978610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:57.979700 (+ 1090us) server_negotiation.cc:407] Connection header received
0808 13:14:57.981553 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:57.981559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:57.981643 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:57.981747 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:57.982893 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.983691 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.985340 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.985975 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.986109 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:57.987656 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:57.987662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:57.987667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:57.987689 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:57.987742 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:57.987758 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:57.987957 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:57.988138 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 13:14:57.988238 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":245,"threads_started":1}
I20260808 13:14:57.988726 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--builtin_ntp_servers=127.31.59.148:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:14:57.989542 18638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:57.978630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40365 (local address 127.31.59.129:42269)
0808 13:14:57.979543 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:57.979584 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:57.981250 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:57.981986 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:57.981995 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:57.982080 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:57.982666 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:57.982690 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.983834 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.983838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:57.985173 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:57.985186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:57.986256 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:57.986266 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:57.986504 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:57.987282 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:14:57.987286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:57.987342 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:14:57.987444 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:57.987939 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:57.987948 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:57.987953 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:14:57.988098 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 13:14:57.988397 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":306,"threads_started":1}
I20260808 13:14:57.991089 18634 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40365
I20260808 13:14:57.991377 18634 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:57.992115 18634 heartbeater.cc:511] Master 127.31.59.190:40365 requested a full tablet report, sending...
I20260808 13:14:57.994347 18428 ts_manager.cc:194] Registered new tserver with Master: 4b3c2c3595d046ff85e3e4945f695a4c (127.31.59.129:36379)
I20260808 13:14:57.996205 18428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.129:42269
W20260808 13:14:58.330593 18639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:58.330888 18639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:58.331020 18639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:58.338284 18639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:58.338397 18639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:58.338541 18639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:14:58.347766 18639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:58.349066 18639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:58.350925 18639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:58.363734 18645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:58.363844 18646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:58.363863 18649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:58.364964 18639 server_base.cc:1034] running on GCE node
I20260808 13:14:58.365612 18639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:58.366551 18639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:58.367805 18639 hybrid_clock.cc:648] HybridClock initialized: now 1786194898367756 us; error 34 us; skew 500 ppm
I20260808 13:14:58.368130 18639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:14:58.370496 18639 webserver.cc:460] Webserver started at http://127.31.59.130:35433/ using document root <none> and password file <none>
I20260808 13:14:58.371173 18639 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:58.371290 18639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:58.371600 18639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:58.374539 18639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "57b8bf84eaea4c70b1ad9f2856249fa6"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "efbce5f96324ce273b9f1998e05fe4e8"
server_key_iv: "f08139b84be9fa450249ffdac0901f52"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.375324 18639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "57b8bf84eaea4c70b1ad9f2856249fa6"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "efbce5f96324ce273b9f1998e05fe4e8"
server_key_iv: "f08139b84be9fa450249ffdac0901f52"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.380539 18639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:14:58.384109 18655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.385427 18639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:14:58.385628 18639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "57b8bf84eaea4c70b1ad9f2856249fa6"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "efbce5f96324ce273b9f1998e05fe4e8"
server_key_iv: "f08139b84be9fa450249ffdac0901f52"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.385838 18639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:58.402261 18639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:58.403105 18639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:58.403403 18639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:58.404385 18639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:58.406001 18639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:58.406127 18639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.406239 18639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:58.406309 18639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.441565 18639 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:44865
I20260808 13:14:58.441612 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:44865 every 8 connection(s)
I20260808 13:14:58.442843 18639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:14:58.449380 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18639
I20260808 13:14:58.449584 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:14:58.450070 31982 external_mini_cluster.cc:1428] Setting key c596cfd3490ee40d11b533b2ca75cec2
I20260808 13:14:58.452945 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--builtin_ntp_servers=127.31.59.148:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:14:58.455472 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.445376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:56385 (local address 127.31.59.190:40365)
0808 13:14:58.445588 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.445593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.446990 (+ 1397us) server_negotiation.cc:407] Connection header received
0808 13:14:58.448782 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.448789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.448872 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.448960 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:58.450138 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.450999 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.452692 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.453082 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.453189 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.454800 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:58.454805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:58.454808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:58.454825 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:58.454859 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:58.454870 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:58.454936 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.455162 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.455249 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:14:58.456797 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.445910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40365 (local address 127.31.59.130:56385)
0808 13:14:58.446831 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:58.446877 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:58.448515 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:58.449202 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:58.449211 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:58.449299 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:58.449971 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.449990 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.451153 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.451157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.452552 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.452565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.453321 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.453328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.453653 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.454452 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:14:58.454455 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:58.454507 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:14:58.454614 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:58.454955 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:58.454965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:58.454974 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:14:58.455147 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 13:14:58.455532 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":301,"threads_started":1}
I20260808 13:14:58.458158 18777 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40365
I20260808 13:14:58.458464 18777 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:58.459355 18777 heartbeater.cc:511] Master 127.31.59.190:40365 requested a full tablet report, sending...
I20260808 13:14:58.460813 18428 ts_manager.cc:194] Registered new tserver with Master: 57b8bf84eaea4c70b1ad9f2856249fa6 (127.31.59.130:44865)
I20260808 13:14:58.461618 18428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.130:56385
W20260808 13:14:58.785988 18780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:14:58.786309 18780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:14:58.786408 18780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:14:58.793785 18780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:14:58.793889 18780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:14:58.794001 18780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:14:58.803018 18780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:14:58.804337 18780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:14:58.806155 18780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:14:58.818897 18787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:58.818919 18790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:14:58.818897 18786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:14:58.819970 18780 server_base.cc:1034] running on GCE node
I20260808 13:14:58.820626 18780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:14:58.821548 18780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:14:58.822747 18780 hybrid_clock.cc:648] HybridClock initialized: now 1786194898822702 us; error 35 us; skew 500 ppm
I20260808 13:14:58.823040 18780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:14:58.825341 18780 webserver.cc:460] Webserver started at http://127.31.59.131:42189/ using document root <none> and password file <none>
I20260808 13:14:58.826051 18780 fs_manager.cc:359] Metadata directory not provided
I20260808 13:14:58.826148 18780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:14:58.826469 18780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:14:58.829345 18780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "a7906c5fb9084e828af4acb4cc3e2a9b"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "eafa8302e42994a3253ad31fe16dd350"
server_key_iv: "330321639067f8e15369a364c95cb348"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.830193 18780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "a7906c5fb9084e828af4acb4cc3e2a9b"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "eafa8302e42994a3253ad31fe16dd350"
server_key_iv: "330321639067f8e15369a364c95cb348"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.835453 18780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:14:58.838886 18796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.840162 18780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:14:58.840324 18780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "a7906c5fb9084e828af4acb4cc3e2a9b"
format_stamp: "Formatted at 2026-08-08 13:14:58 on dist-test-slave-49h3"
server_key: "eafa8302e42994a3253ad31fe16dd350"
server_key_iv: "330321639067f8e15369a364c95cb348"
server_key_version: "encryptionkey@0"
I20260808 13:14:58.840528 18780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:14:58.853158 18780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:14:58.853967 18780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:14:58.854251 18780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:14:58.855146 18780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:14:58.856545 18780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:14:58.856642 18780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.856745 18780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:14:58.856813 18780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:14:58.887580 18780 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:42217
I20260808 13:14:58.887634 18924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:42217 every 8 connection(s)
I20260808 13:14:58.888906 18780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:14:58.895362 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18780
I20260808 13:14:58.895526 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:14:58.895957 31982 external_mini_cluster.cc:1428] Setting key c0d0a928ce03be890f10f935cb47f97a
I20260808 13:14:58.901319 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.891631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:51523 (local address 127.31.59.190:40365)
0808 13:14:58.891819 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.891823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.893133 (+ 1310us) server_negotiation.cc:407] Connection header received
0808 13:14:58.894945 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.894951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.895054 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.895140 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:58.896294 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.897285 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.899026 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.899444 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.899538 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.900725 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:58.900730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:58.900733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:58.900749 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:58.900788 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:58.900799 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:58.900859 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.901009 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.901084 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:14:58.902313 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.892084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:40365 (local address 127.31.59.131:51523)
0808 13:14:58.892969 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:58.893013 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:58.894681 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:58.895391 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:58.895399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:58.895486 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:14:58.896135 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.896154 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.897460 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.897466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.898897 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.898910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.899609 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.899613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.899813 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.900470 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:14:58.900473 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:14:58.900510 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:14:58.900581 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:14:58.900862 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:14:58.900868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:14:58.900873 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:14:58.900995 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:14:58.901269 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":280,"threads_started":1}
I20260808 13:14:58.903657 18925 heartbeater.cc:347] Connected to a master server at 127.31.59.190:40365
I20260808 13:14:58.903934 18925 heartbeater.cc:464] Registering TS with master...
I20260808 13:14:58.904644 18925 heartbeater.cc:511] Master 127.31.59.190:40365 requested a full tablet report, sending...
I20260808 13:14:58.905987 18428 ts_manager.cc:194] Registered new tserver with Master: a7906c5fb9084e828af4acb4cc3e2a9b (127.31.59.131:42217)
I20260808 13:14:58.906773 18428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.131:51523
I20260808 13:14:58.910941 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:14:58.924659 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.919604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.31.59.190:40365)
0808 13:14:58.919834 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.919839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.919860 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:14:58.919962 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.919967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.920044 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.920123 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:58.920678 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.921406 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.922686 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.923093 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.923181 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.924103 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:58.924107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:58.924110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:58.924126 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:58.924161 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:58.924172 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:58.924235 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.924342 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.924419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:14:58.928440 18428 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:14:58.935341 18428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:14:58.958752 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.948102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42314)
0808 13:14:58.948858 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:58.948894 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:58.949086 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:58.950324 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:58.950333 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:58.950368 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:58.950832 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.950845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.953245 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.953252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.956041 (+ 2789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.956062 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.957635 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.957642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.957915 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.957974 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 13:14:58.958114 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 13:14:58.958296 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":62,"spinlock_wait_cycles":33024,"thread_start_us":295,"threads_started":1}
I20260808 13:14:58.959887 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.948516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.31.59.129:36379)
0808 13:14:58.949660 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.949670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.949702 (+ 32us) server_negotiation.cc:407] Connection header received
0808 13:14:58.949806 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.949814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.950084 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.951115 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:58.951417 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.953087 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.956253 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.957427 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.957585 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.957851 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.959429 (+ 1578us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.959636 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":852,"threads_started":1}
I20260808 13:14:58.961566 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.947523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44865 (local address 127.0.0.1:55224)
0808 13:14:58.948856 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:58.948901 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:58.949087 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:58.954219 (+ 5132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:58.954223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:58.954246 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:58.955031 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.955044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.956517 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.956523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.958584 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.958598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.960735 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.960741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.960906 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.960938 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:14:58.961035 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:14:58.961158 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"spinlock_wait_cycles":5504,"thread_start_us":470,"threads_started":1}
I20260808 13:14:58.961637 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.947690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.31.59.130:44865)
0808 13:14:58.953641 (+ 5951us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.953650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.953680 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:14:58.953774 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.953782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.954016 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.954689 (+ 673us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:58.955205 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.956374 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.958831 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.960536 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.960672 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.960889 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.961087 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.961267 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5801,"thread_start_us":5651,"threads_started":1}
I20260808 13:14:58.962404 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.949356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.31.59.131:42217)
0808 13:14:58.950083 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:58.950092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:58.954275 (+ 4183us) server_negotiation.cc:407] Connection header received
0808 13:14:58.954378 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:58.954386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:58.954610 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:58.954794 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:58.955658 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.957058 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.959496 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.960822 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.960945 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.961179 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:58.961885 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 13:14:58.962039 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":420,"threads_started":1}
I20260808 13:14:58.962899 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:58.948812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:42217 (local address 127.0.0.1:45716)
0808 13:14:58.954000 (+ 5188us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:58.954018 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:58.954143 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:58.955006 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:58.955010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:58.955038 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:58.955431 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.955445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.957216 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.957221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.959311 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:58.959326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:58.961307 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:58.961313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:58.961522 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:58.961555 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:14:58.961641 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 13:14:58.962662 (+ 1021us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5076,"mutex_wait_us":23,"thread_start_us":4921,"threads_started":1}
I20260808 13:14:58.963711 18563 tablet_service.cc:1467] Processing CreateTablet for tablet b64258a5b0b64d28bf2c43a369f94801 (DEFAULT_TABLE table=test-table [id=b42c9a43ec7c4da58c577c8d21698356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:58.965266 18563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64258a5b0b64d28bf2c43a369f94801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:58.965077 18710 tablet_service.cc:1467] Processing CreateTablet for tablet b64258a5b0b64d28bf2c43a369f94801 (DEFAULT_TABLE table=test-table [id=b42c9a43ec7c4da58c577c8d21698356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:58.965950 18850 tablet_service.cc:1467] Processing CreateTablet for tablet b64258a5b0b64d28bf2c43a369f94801 (DEFAULT_TABLE table=test-table [id=b42c9a43ec7c4da58c577c8d21698356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:14:58.966537 18710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64258a5b0b64d28bf2c43a369f94801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:58.967389 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64258a5b0b64d28bf2c43a369f94801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:14:58.987663 18945 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Bootstrap starting.
I20260808 13:14:58.989107 18944 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Bootstrap starting.
I20260808 13:14:58.989707 18946 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: Bootstrap starting.
I20260808 13:14:58.992666 18944 tablet_bootstrap.cc:654] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:58.993525 18946 tablet_bootstrap.cc:654] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:58.993655 18945 tablet_bootstrap.cc:654] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Neither blocks nor log segments found. Creating new log.
I20260808 13:14:58.993834 18944 log.cc:824] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:58.994760 18945 log.cc:824] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:58.994760 18946 log.cc:824] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:14:58.997119 18944 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: No bootstrap required, opened a new log
I20260808 13:14:58.997377 18944 ts_tablet_manager.cc:1397] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 13:14:58.997608 18946 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: No bootstrap required, opened a new log
I20260808 13:14:58.997874 18946 ts_tablet_manager.cc:1397] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260808 13:14:58.998700 18945 tablet_bootstrap.cc:492] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: No bootstrap required, opened a new log
I20260808 13:14:58.999190 18634 heartbeater.cc:503] Master 127.31.59.190:40365 was elected leader, sending a full tablet report...
I20260808 13:14:59.000695 18945 ts_tablet_manager.cc:1397] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.005s
I20260808 13:14:59.006219 18944 raft_consensus.cc:348] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.006608 18944 raft_consensus.cc:374] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:59.006737 18944 raft_consensus.cc:729] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b8bf84eaea4c70b1ad9f2856249fa6, State: Initialized, Role: FOLLOWER
I20260808 13:14:59.007323 18944 consensus_queue.cc:260] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.008371 18777 heartbeater.cc:503] Master 127.31.59.190:40365 was elected leader, sending a full tablet report...
I20260808 13:14:59.009248 18946 raft_consensus.cc:348] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.009809 18946 raft_consensus.cc:374] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:59.009948 18946 raft_consensus.cc:729] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3c2c3595d046ff85e3e4945f695a4c, State: Initialized, Role: FOLLOWER
I20260808 13:14:59.010295 18944 ts_tablet_manager.cc:1428] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Time spent starting tablet: real 0.013s user 0.006s sys 0.004s
I20260808 13:14:59.010669 18946 consensus_queue.cc:260] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.011246 18945 raft_consensus.cc:348] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.012563 18946 ts_tablet_manager.cc:1428] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260808 13:14:59.012460 18945 raft_consensus.cc:374] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:14:59.013628 18945 raft_consensus.cc:729] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7906c5fb9084e828af4acb4cc3e2a9b, State: Initialized, Role: FOLLOWER
I20260808 13:14:59.014783 18945 consensus_queue.cc:260] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.015861 18925 heartbeater.cc:503] Master 127.31.59.190:40365 was elected leader, sending a full tablet report...
I20260808 13:14:59.016439 18945 ts_tablet_manager.cc:1428] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Time spent starting tablet: real 0.016s user 0.012s sys 0.000s
I20260808 13:14:59.081288 18953 raft_consensus.cc:482] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:14:59.081645 18953 raft_consensus.cc:504] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.083496 18953 leader_election.cc:288] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b8bf84eaea4c70b1ad9f2856249fa6 (127.31.59.130:44865), 4b3c2c3595d046ff85e3e4945f695a4c (127.31.59.129:36379)
I20260808 13:14:59.089389 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.084015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:50579 (local address 127.31.59.129:36379)
0808 13:14:59.084230 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.084236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.084460 (+ 224us) server_negotiation.cc:407] Connection header received
0808 13:14:59.084649 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.084654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.084758 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.084843 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:59.085234 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.085963 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.087690 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.088481 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.088622 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.088692 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.088963 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.089100 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:14:59.089403 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.084008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.31.59.131:50579)
0808 13:14:59.084351 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.084367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:59.084508 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:59.084893 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:59.084898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:59.084925 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:59.085095 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:59.085103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.086177 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.086181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:59.087538 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:59.087550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.088625 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.088631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:59.088798 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.088822 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:14:59.088902 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 13:14:59.089045 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260808 13:14:59.090284 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.084071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:44865 (local address 127.31.59.131:55503)
0808 13:14:59.084877 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.084894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:14:59.085028 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:14:59.085351 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:14:59.085356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:14:59.085380 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:14:59.085719 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:59.085727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.086776 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.086780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:59.088117 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:14:59.088128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.089489 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.089495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:14:59.089669 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.089701 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:14:59.089847 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 13:14:59.089979 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":438,"threads_started":1}
I20260808 13:14:59.090299 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.084037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:55503 (local address 127.31.59.130:44865)
0808 13:14:59.084230 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.084235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.084982 (+ 747us) server_negotiation.cc:407] Connection header received
0808 13:14:59.085162 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.085166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.085236 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.085313 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:14:59.085879 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.086625 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.088299 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.089159 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.089291 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.089346 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.089876 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.090030 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:14:59.090706 18585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64258a5b0b64d28bf2c43a369f94801" candidate_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" is_pre_election: true
I20260808 13:14:59.091243 18585 raft_consensus.cc:2454] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7906c5fb9084e828af4acb4cc3e2a9b in term 0.
I20260808 13:14:59.091239 18730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64258a5b0b64d28bf2c43a369f94801" candidate_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" is_pre_election: true
I20260808 13:14:59.091717 18730 raft_consensus.cc:2454] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7906c5fb9084e828af4acb4cc3e2a9b in term 0.
I20260808 13:14:59.091933 18798 leader_election.cc:302] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3c2c3595d046ff85e3e4945f695a4c, a7906c5fb9084e828af4acb4cc3e2a9b; no voters:
I20260808 13:14:59.092351 18953 raft_consensus.cc:2781] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:14:59.092495 18953 raft_consensus.cc:482] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:14:59.092605 18953 raft_consensus.cc:3037] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:59.094861 18953 raft_consensus.cc:504] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.095432 18953 leader_election.cc:288] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [CANDIDATE]: Term 1 election: Requested vote from peers 57b8bf84eaea4c70b1ad9f2856249fa6 (127.31.59.130:44865), 4b3c2c3595d046ff85e3e4945f695a4c (127.31.59.129:36379)
I20260808 13:14:59.095852 18730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64258a5b0b64d28bf2c43a369f94801" candidate_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6"
I20260808 13:14:59.095894 18585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64258a5b0b64d28bf2c43a369f94801" candidate_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3c2c3595d046ff85e3e4945f695a4c"
I20260808 13:14:59.096024 18730 raft_consensus.cc:3037] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:59.096087 18585 raft_consensus.cc:3037] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:14:59.098340 18585 raft_consensus.cc:2454] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7906c5fb9084e828af4acb4cc3e2a9b in term 1.
I20260808 13:14:59.098351 18730 raft_consensus.cc:2454] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7906c5fb9084e828af4acb4cc3e2a9b in term 1.
I20260808 13:14:59.098747 18800 leader_election.cc:302] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57b8bf84eaea4c70b1ad9f2856249fa6, a7906c5fb9084e828af4acb4cc3e2a9b; no voters:
I20260808 13:14:59.099045 18953 raft_consensus.cc:2781] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:14:59.099488 18953 raft_consensus.cc:686] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 1 LEADER]: Becoming Leader. State: Replica: a7906c5fb9084e828af4acb4cc3e2a9b, State: Running, Role: LEADER
I20260808 13:14:59.099927 18953 consensus_queue.cc:237] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } }
I20260808 13:14:59.104331 18427 catalog_manager.cc:5495] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b reported cstate change: term changed from 0 to 1, leader changed from <none> to a7906c5fb9084e828af4acb4cc3e2a9b (127.31.59.131). New cstate: current_term: 1 leader_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7906c5fb9084e828af4acb4cc3e2a9b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 } health_report { overall_health: UNKNOWN } } }
W20260808 13:14:59.140739 18926 tablet.cc:2369] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:14:59.152947 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.147951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.31.59.131:42217)
0808 13:14:59.148132 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.148138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.148275 (+ 137us) server_negotiation.cc:407] Connection header received
0808 13:14:59.148469 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.148475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.148568 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.148691 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:14:59.148970 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.149763 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.151122 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.151514 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.151598 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.152025 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:14:59.152537 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:14:59.152640 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.152694 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.152756 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 13:14:59.161204 18730 raft_consensus.cc:1261] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 1 FOLLOWER]: Refusing update from remote peer a7906c5fb9084e828af4acb4cc3e2a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:59.161204 18585 raft_consensus.cc:1261] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 1 FOLLOWER]: Refusing update from remote peer a7906c5fb9084e828af4acb4cc3e2a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:14:59.162268 18955 consensus_queue.cc:1035] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3c2c3595d046ff85e3e4945f695a4c" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:59.162660 18953 consensus_queue.cc:1035] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b8bf84eaea4c70b1ad9f2856249fa6" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:14:59.173549 18958 mvcc.cc:205] Tried to move back new op lower bound from 7316254306953273344 to 7316254306717073408. Current Snapshot: MvccSnapshot[applied={T|T < 7316254306953273344}]
I20260808 13:14:59.174760 18959 mvcc.cc:205] Tried to move back new op lower bound from 7316254306953273344 to 7316254306717073408. Current Snapshot: MvccSnapshot[applied={T|T < 7316254306953273344}]
I20260808 13:14:59.186489 18428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:59.186803 18428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:14:59.190593 18428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 442210998529495788d92b1a9b978dae: Sending DeleteTablet for 3 replicas of tablet b64258a5b0b64d28bf2c43a369f94801
I20260808 13:14:59.191715 18710 tablet_service.cc:1514] Processing DeleteTablet for tablet b64258a5b0b64d28bf2c43a369f94801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:59 UTC) from {username='slave'} at 127.0.0.1:55224
I20260808 13:14:59.191936 18563 tablet_service.cc:1514] Processing DeleteTablet for tablet b64258a5b0b64d28bf2c43a369f94801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:59 UTC) from {username='slave'} at 127.0.0.1:42314
I20260808 13:14:59.192122 18850 tablet_service.cc:1514] Processing DeleteTablet for tablet b64258a5b0b64d28bf2c43a369f94801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:14:59 UTC) from {username='slave'} at 127.0.0.1:45716
I20260808 13:14:59.198032 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18492
I20260808 13:14:59.206713 18971 tablet_replica.cc:330] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: stopping tablet replica
I20260808 13:14:59.207252 18971 raft_consensus.cc:2229] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:14:59.207995 18971 raft_consensus.cc:2258] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:59.212677 18971 ts_tablet_manager.cc:1905] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:59.217120 18969 tablet_replica.cc:330] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: stopping tablet replica
I20260808 13:14:59.218104 18970 tablet_replica.cc:330] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c: stopping tablet replica
I20260808 13:14:59.218569 18970 raft_consensus.cc:2229] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:59.219178 18970 raft_consensus.cc:2258] T b64258a5b0b64d28bf2c43a369f94801 P 4b3c2c3595d046ff85e3e4945f695a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:59.219419 18969 raft_consensus.cc:2229] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:14:59.220290 18969 raft_consensus.cc:2258] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:14:59.221956 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.199123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.31.59.129:36379)
0808 13:14:59.199315 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.199321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.211533 (+ 12212us) server_negotiation.cc:407] Connection header received
0808 13:14:59.212064 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.212070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.212231 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.212316 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:59.213070 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.214328 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.217355 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.218093 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.218211 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.220152 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:59.220164 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:59.220180 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:59.220976 (+ 796us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:59.221052 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:59.221078 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:59.221194 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.221489 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.221609 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:14:59.222458 18971 ts_tablet_manager.cc:1918] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:59.222644 18971 log.cc:1196] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/b64258a5b0b64d28bf2c43a369f94801
I20260808 13:14:59.222684 18629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:59.223224 18971 ts_tablet_manager.cc:1939] T b64258a5b0b64d28bf2c43a369f94801 P a7906c5fb9084e828af4acb4cc3e2a9b: Deleting consensus metadata
I20260808 13:14:59.223590 18969 ts_tablet_manager.cc:1905] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:14:59.225009 18414 catalog_manager.cc:4841] TS a7906c5fb9084e828af4acb4cc3e2a9b (127.31.59.131:42217): tablet b64258a5b0b64d28bf2c43a369f94801 (table test-table [id=b42c9a43ec7c4da58c577c8d21698356]) successfully deleted
I20260808 13:14:59.228950 18969 ts_tablet_manager.cc:1918] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:14:59.229097 18969 log.cc:1196] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/b64258a5b0b64d28bf2c43a369f94801
I20260808 13:14:59.229652 18969 ts_tablet_manager.cc:1939] T b64258a5b0b64d28bf2c43a369f94801 P 57b8bf84eaea4c70b1ad9f2856249fa6: Deleting consensus metadata
I20260808 13:14:59.231542 18414 catalog_manager.cc:4841] TS 57b8bf84eaea4c70b1ad9f2856249fa6 (127.31.59.130:44865): tablet b64258a5b0b64d28bf2c43a369f94801 (table test-table [id=b42c9a43ec7c4da58c577c8d21698356]) successfully deleted
I20260808 13:14:59.318509 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18492
W20260808 13:14:59.339217 18412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:36379 (error 108)
I20260808 13:14:59.340633 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18639
I20260808 13:14:59.341158 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.340665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42324)
0808 13:14:59.340851 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.340984 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 13:14:59.341420 18412 catalog_manager.cc:4584] TS 4b3c2c3595d046ff85e3e4945f695a4c (127.31.59.129:36379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b64258a5b0b64d28bf2c43a369f94801: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
I20260808 13:14:59.347985 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.341469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.31.59.130:44865)
0808 13:14:59.341668 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.341674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.341912 (+ 238us) server_negotiation.cc:407] Connection header received
0808 13:14:59.341977 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.341981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.342080 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.342175 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:59.342605 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.343327 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.344806 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.345227 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.345358 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.346592 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:59.346601 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:59.346614 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:59.347236 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:59.347290 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:59.347310 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:59.347409 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.347540 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.347654 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:14:59.348708 18772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:59.395320 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.394751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42332)
0808 13:14:59.395000 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.395117 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 13:14:59.396528 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.396108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42344)
0808 13:14:59.396272 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.396387 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:14:59.439210 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18639
I20260808 13:14:59.457242 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.456658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42356)
0808 13:14:59.456900 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.457015 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 13:14:59.458480 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.458042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42366)
0808 13:14:59.458227 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.458326 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:14:59.459733 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18780
I20260808 13:14:59.467764 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.460411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.31.59.131:42217)
0808 13:14:59.460584 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:14:59.460590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:14:59.460715 (+ 125us) server_negotiation.cc:407] Connection header received
0808 13:14:59.460909 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:14:59.460913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:14:59.460990 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:14:59.461077 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:14:59.461579 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.462799 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.464087 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:14:59.464786 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:14:59.464955 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:14:59.466044 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:14:59.466058 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:14:59.466082 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:14:59.466906 (+ 824us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:14:59.466967 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:14:59.467010 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:14:59.467121 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:14:59.467269 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 13:14:59.467375 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 13:14:59.468446 18920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:59.539928 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.539340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42372)
0808 13:14:59.539590 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.539711 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 13:14:59.541152 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.540686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42374)
0808 13:14:59.540915 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.541021 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260808 13:14:59.593581 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18780
I20260808 13:14:59.614665 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18394
I20260808 13:14:59.615767 18458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:14:59.727211 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:14:59.726465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36379 (local address 127.0.0.1:42390)
0808 13:14:59.726888 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 13:14:59.727006 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269}
I20260808 13:14:59.727319 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18394
2026-08-08T13:14:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:14:59 dist-test-slave-49h3 krb5kdc[18982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:14:59 dist-test-slave-49h3 krb5kdc[18982](info): set up 2 sockets
Aug 08 13:14:59 dist-test-slave-49h3 krb5kdc[18982](info): commencing operation
krb5kdc: starting...
W20260808 13:15:00.821266 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:01 dist-test-slave-49h3 krb5kdc[18982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:01.136929 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33777
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:33777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:01.472055 18996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:01.472397 18996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:01.472501 18996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:01.479647 18996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:01.479794 18996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:01.479852 18996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:01.479912 18996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:01.479955 18996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:01.489825 18996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:33777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:01.491050 18996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:01.492290 18996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:01.503453 19005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:01.503523 19002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:01.503500 19003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:01.504791 18996 server_base.cc:1034] running on GCE node
I20260808 13:15:01.505622 18996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:01.507050 18996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:01.508255 18996 hybrid_clock.cc:648] HybridClock initialized: now 1786194901508190 us; error 52 us; skew 500 ppm
I20260808 13:15:01.508586 18996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:01 dist-test-slave-49h3 krb5kdc[18982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:01.511621 18996 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:01.513128 18996 webserver.cc:460] Webserver started at http://127.31.59.190:42271/ using document root <none> and password file <none>
I20260808 13:15:01.513852 18996 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:01.513978 18996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:01.514288 18996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:01.517297 18996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "ca9382158f204c0bbbcd5a2a4bd82652"
format_stamp: "Formatted at 2026-08-08 13:15:01 on dist-test-slave-49h3"
server_key: "5c4978afa5bdb92e2d93c0f312ceedfd"
server_key_iv: "30e40a7416fb48393063b7e601a8654a"
server_key_version: "encryptionkey@0"
I20260808 13:15:01.518100 18996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "ca9382158f204c0bbbcd5a2a4bd82652"
format_stamp: "Formatted at 2026-08-08 13:15:01 on dist-test-slave-49h3"
server_key: "5c4978afa5bdb92e2d93c0f312ceedfd"
server_key_iv: "30e40a7416fb48393063b7e601a8654a"
server_key_version: "encryptionkey@0"
I20260808 13:15:01.523350 18996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:15:01.526831 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:01.528132 18996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:01.528303 18996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "ca9382158f204c0bbbcd5a2a4bd82652"
format_stamp: "Formatted at 2026-08-08 13:15:01 on dist-test-slave-49h3"
server_key: "5c4978afa5bdb92e2d93c0f312ceedfd"
server_key_iv: "30e40a7416fb48393063b7e601a8654a"
server_key_version: "encryptionkey@0"
I20260808 13:15:01.528489 18996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:01.562969 18996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:01.567564 18996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:01.567862 18996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:01.585726 18996 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:33777
I20260808 13:15:01.585747 19065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:33777 every 8 connection(s)
I20260808 13:15:01.587126 18996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:01.590816 19066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:01.595875 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 18996
I20260808 13:15:01.596042 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:01.596451 31982 external_mini_cluster.cc:1428] Setting key 766352858f97930407b9ead938e4c7d7
I20260808 13:15:01.601406 19066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: Bootstrap starting.
I20260808 13:15:01.605069 19066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:01.606387 19066 log.cc:824] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:01.609936 19066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: No bootstrap required, opened a new log
Aug 08 13:15:01 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:01.618178 19066 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } }
I20260808 13:15:01.618597 19066 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:01.618710 19066 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9382158f204c0bbbcd5a2a4bd82652, State: Initialized, Role: FOLLOWER
I20260808 13:15:01.619343 19066 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } }
I20260808 13:15:01.619596 19066 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:01.619750 19066 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:01.619921 19066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:01.620460 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:01.598259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.31.59.190:33777)
0808 13:15:01.599108 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:01.599119 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:01.599164 (+ 45us) server_negotiation.cc:407] Connection header received
0808 13:15:01.600885 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:01.600906 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:01.601268 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:01.601638 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:01.603428 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:01.604707 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:01.606628 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:01.607657 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:01.607846 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:01.612520 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:01.612528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:01.612542 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:01.612591 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:01.615790 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:01.616552 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:01.616560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:01.616568 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:01.616757 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:01.617163 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:01.617167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:01.617169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:01.617828 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:01.618124 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:01.618642 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 13:15:01.619224 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":381,"threads_started":1}
I20260808 13:15:01.622478 19066 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } }
I20260808 13:15:01.622992 19066 leader_election.cc:302] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca9382158f204c0bbbcd5a2a4bd82652; no voters:
I20260808 13:15:01.623616 19066 leader_election.cc:288] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:01.623739 19073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:01.624430 19073 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [term 1 LEADER]: Becoming Leader. State: Replica: ca9382158f204c0bbbcd5a2a4bd82652, State: Running, Role: LEADER
I20260808 13:15:01.624991 19073 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } }
I20260808 13:15:01.625608 19066 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:01.627843 19075 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9382158f204c0bbbcd5a2a4bd82652. Latest consensus state: current_term: 1 leader_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } } }
I20260808 13:15:01.628253 19075 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:01.628373 19074 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9382158f204c0bbbcd5a2a4bd82652" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33777 } } }
I20260808 13:15:01.628713 19074 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:01.634145 19080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:01.639717 19080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:01.649639 19080 catalog_manager.cc:1347] Generated new cluster ID: 754c8a77cf0e4d24b95e081e3593e853
I20260808 13:15:01.649758 19080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:01.673537 19080 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:01.674937 19080 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:01.682003 19080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: Generated new TSK 0
I20260808 13:15:01.682837 19080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:15:01.756841 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:02.105352 19096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:02.105690 19096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:02.105801 19096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:02.113217 19096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:02.113345 19096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:02.113507 19096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:15:02.123729 19096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:02.125073 19096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:02.126575 19096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:02.140043 19104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:02.140174 19101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:02.140153 19102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:02.141109 19096 server_base.cc:1034] running on GCE node
I20260808 13:15:02.141781 19096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:02.142728 19096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:02.143944 19096 hybrid_clock.cc:648] HybridClock initialized: now 1786194902143881 us; error 50 us; skew 500 ppm
I20260808 13:15:02.144259 19096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:02 dist-test-slave-49h3 krb5kdc[18982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194902, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:02.147454 19096 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:15:02.148861 19096 webserver.cc:460] Webserver started at http://127.31.59.129:44095/ using document root <none> and password file <none>
I20260808 13:15:02.149560 19096 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:02.149673 19096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:02.149956 19096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:02.152848 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "5b758e4cc6e7467caf31af6e202c446f"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "995ada17cc484bec94d3aa112db4b945"
server_key_iv: "b56270bbfdcfc805ab6c0af3a915cd80"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.153630 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "5b758e4cc6e7467caf31af6e202c446f"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "995ada17cc484bec94d3aa112db4b945"
server_key_iv: "b56270bbfdcfc805ab6c0af3a915cd80"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.158703 19096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:15:02.162166 19112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.163463 19096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:02.163601 19096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "5b758e4cc6e7467caf31af6e202c446f"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "995ada17cc484bec94d3aa112db4b945"
server_key_iv: "b56270bbfdcfc805ab6c0af3a915cd80"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.163767 19096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:02.200771 19096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:02.204744 19096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:02.205015 19096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:02.206108 19096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:02.207569 19096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:02.207657 19096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.207767 19096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:02.207810 19096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.239127 19096 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:41795
I20260808 13:15:02.239197 19230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:41795 every 8 connection(s)
I20260808 13:15:02.240442 19096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:15:02.249045 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19096
I20260808 13:15:02.249231 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:15:02.249740 31982 external_mini_cluster.cc:1428] Setting key b370f03de66261c6bef9803b079e936f
Aug 08 13:15:02 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194902, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:02.260449 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:02.242974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:52951 (local address 127.31.59.190:33777)
0808 13:15:02.243476 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:02.243482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:02.244350 (+ 868us) server_negotiation.cc:407] Connection header received
0808 13:15:02.246277 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:02.246281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:02.246351 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:02.246438 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:02.247920 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.248704 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.250411 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.250924 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.251073 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:02.254586 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:02.254593 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:02.254597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:02.254640 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:02.257527 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:02.258235 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:02.258241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:02.258244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:02.258387 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:02.258770 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:02.258775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:02.258777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:02.259090 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:02.259227 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:02.259885 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 13:15:02.260105 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":228,"threads_started":1}
I20260808 13:15:02.261373 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:02.243484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33777 (local address 127.31.59.129:52951)
0808 13:15:02.244185 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:02.244229 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:02.245994 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:02.246646 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:02.246654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:02.247181 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:02.247756 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:02.247779 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.248878 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.248882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:02.250266 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:02.250280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.251172 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.251178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:02.251419 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:02.252135 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:02.252158 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:02.254359 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:02.257647 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:02.257654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:02.257667 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:02.258112 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:02.258473 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:02.258476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:02.258478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:02.258661 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:02.259191 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:02.259196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:02.259504 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 13:15:02.259731 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 13:15:02.260136 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":274,"threads_started":1}
I20260808 13:15:02.262841 19231 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33777
I20260808 13:15:02.263166 19231 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:02.264009 19231 heartbeater.cc:511] Master 127.31.59.190:33777 requested a full tablet report, sending...
I20260808 13:15:02.266163 19030 ts_manager.cc:194] Registered new tserver with Master: 5b758e4cc6e7467caf31af6e202c446f (127.31.59.129:41795)
I20260808 13:15:02.268016 19030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:52951
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:15:02.316704 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:02.660723 19238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:02.661006 19238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:02.661155 19238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:02.668695 19238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:02.668813 19238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:02.668962 19238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:15:02.679059 19238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:02.680351 19238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:02.681667 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:02.694927 19244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:02.694965 19243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:02.695003 19246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:02.695968 19238 server_base.cc:1034] running on GCE node
I20260808 13:15:02.696604 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:02.697633 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:02.698873 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786194902698820 us; error 36 us; skew 500 ppm
I20260808 13:15:02.699196 19238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:02 dist-test-slave-49h3 krb5kdc[18982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194902, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:02.707484 19238 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:15:02.709136 19238 webserver.cc:460] Webserver started at http://127.31.59.130:41659/ using document root <none> and password file <none>
I20260808 13:15:02.709895 19238 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:02.710026 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:02.710338 19238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:02.713351 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "f31b874e03f440e397bd376bac1bf3af"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "48ddcf07ce301348826a19bfb115651f"
server_key_iv: "a4139af8423ef3485e8d028bca300935"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.714200 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "f31b874e03f440e397bd376bac1bf3af"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "48ddcf07ce301348826a19bfb115651f"
server_key_iv: "a4139af8423ef3485e8d028bca300935"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.719640 19238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:02.723197 19254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.724505 19238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:15:02.724682 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "f31b874e03f440e397bd376bac1bf3af"
format_stamp: "Formatted at 2026-08-08 13:15:02 on dist-test-slave-49h3"
server_key: "48ddcf07ce301348826a19bfb115651f"
server_key_iv: "a4139af8423ef3485e8d028bca300935"
server_key_version: "encryptionkey@0"
I20260808 13:15:02.724896 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:02.739907 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:02.743960 19238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:02.744273 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:02.745283 19238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:02.746760 19238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:02.746850 19238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.746973 19238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:02.747033 19238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:02.778748 19238 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:36943
I20260808 13:15:02.778798 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:36943 every 8 connection(s)
I20260808 13:15:02.780061 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:15:02.787439 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19238
I20260808 13:15:02.787638 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:15:02.788094 31982 external_mini_cluster.cc:1428] Setting key 62f7e52de41a3962a84033959b3f4f35
Aug 08 13:15:02 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194902, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:02.799575 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:02.782604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:39173 (local address 127.31.59.190:33777)
0808 13:15:02.783051 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:02.783055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:02.783968 (+ 913us) server_negotiation.cc:407] Connection header received
0808 13:15:02.785818 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:02.785824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:02.785904 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:02.786003 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:02.787553 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.788375 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.789934 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.790375 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.790485 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:02.794085 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:02.794090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:02.794093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:02.794125 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:02.796485 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:02.797182 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:02.797186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:02.797188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:02.797327 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:02.797786 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:02.797789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:02.797791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:02.798063 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:02.798199 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:02.798895 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 13:15:02.799155 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":221,"threads_started":1}
I20260808 13:15:02.800480 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:02.783111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33777 (local address 127.31.59.130:39173)
0808 13:15:02.783807 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:02.783849 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:02.785553 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:02.786261 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:02.786269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:02.786764 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:02.787378 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:02.787398 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.788531 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.788535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:02.789799 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:02.789811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:02.790590 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:02.790597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:02.790814 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:02.791472 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:02.791494 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:02.793869 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:02.796630 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:02.796639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:02.796655 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:02.797063 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:02.797432 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:02.797437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:02.797441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:02.797668 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:02.798191 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:02.798199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:02.798549 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 13:15:02.798778 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 13:15:02.799339 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":269,"threads_started":1}
I20260808 13:15:02.801899 19374 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33777
I20260808 13:15:02.802181 19374 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:02.802891 19374 heartbeater.cc:511] Master 127.31.59.190:33777 requested a full tablet report, sending...
I20260808 13:15:02.804378 19030 ts_manager.cc:194] Registered new tserver with Master: f31b874e03f440e397bd376bac1bf3af (127.31.59.130:36943)
I20260808 13:15:02.805510 19030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:39173
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:15:02.862653 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:03.206965 19380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:03.207295 19380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:03.207445 19380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:03.214764 19380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:03.214900 19380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:03.215067 19380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:15:03.226193 19380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:03.227882 19380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:03.229737 19380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:03.243469 19389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:03.243427 19386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:03.243546 19387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:03.245611 19380 server_base.cc:1034] running on GCE node
I20260808 13:15:03.246284 19380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:03.247215 19380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:03.248425 19380 hybrid_clock.cc:648] HybridClock initialized: now 1786194903248380 us; error 33 us; skew 500 ppm
I20260808 13:15:03.248728 19380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:03 dist-test-slave-49h3 krb5kdc[18982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194903, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:03.251776 19380 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:15:03.253206 19380 webserver.cc:460] Webserver started at http://127.31.59.131:39917/ using document root <none> and password file <none>
I20260808 13:15:03.253944 19380 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:03.254063 19380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:03.254370 19380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:03.257364 19380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "b5d07336bac04f81adee6896ec5ad594"
format_stamp: "Formatted at 2026-08-08 13:15:03 on dist-test-slave-49h3"
server_key: "6926447610c6ed988801e0bf68f7dfa6"
server_key_iv: "11798bc6f082b35ec435074d54501b4a"
server_key_version: "encryptionkey@0"
I20260808 13:15:03.258163 19380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "b5d07336bac04f81adee6896ec5ad594"
format_stamp: "Formatted at 2026-08-08 13:15:03 on dist-test-slave-49h3"
server_key: "6926447610c6ed988801e0bf68f7dfa6"
server_key_iv: "11798bc6f082b35ec435074d54501b4a"
server_key_version: "encryptionkey@0"
I20260808 13:15:03.263420 19380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:03.266932 19397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:03.268361 19380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:03.268510 19380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "b5d07336bac04f81adee6896ec5ad594"
format_stamp: "Formatted at 2026-08-08 13:15:03 on dist-test-slave-49h3"
server_key: "6926447610c6ed988801e0bf68f7dfa6"
server_key_iv: "11798bc6f082b35ec435074d54501b4a"
server_key_version: "encryptionkey@0"
I20260808 13:15:03.268693 19380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:03.271188 19231 heartbeater.cc:503] Master 127.31.59.190:33777 was elected leader, sending a full tablet report...
I20260808 13:15:03.295018 19380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:03.299332 19380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:03.299590 19380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:03.300541 19380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:03.302260 19380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:03.302340 19380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:03.302441 19380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:03.302484 19380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:03.333981 19380 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:37299
I20260808 13:15:03.334031 19525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:37299 every 8 connection(s)
I20260808 13:15:03.335318 19380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:15:03.337739 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19380
I20260808 13:15:03.337960 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:15:03.338593 31982 external_mini_cluster.cc:1428] Setting key 430c6e5c3aecc7b2a22bca9542ddf58c
Aug 08 13:15:03 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194903, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:03.355924 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.337910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:60979 (local address 127.31.59.190:33777)
0808 13:15:03.341824 (+ 3914us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.341830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.341846 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:15:03.341901 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.341905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.341984 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.342084 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:03.343612 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.344390 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.345964 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.346384 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.346496 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.349868 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:03.349874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:03.349878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:03.349921 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:03.352518 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.353395 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.353400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.353402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.353587 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.354061 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.354067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.354071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.354456 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:03.354605 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.355319 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.355552 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3796,"thread_start_us":3690,"threads_started":1}
I20260808 13:15:03.356884 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.338387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:33777 (local address 127.31.59.131:60979)
0808 13:15:03.339260 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.339306 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.341265 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.342344 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.342352 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.342852 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:03.343433 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.343451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.344534 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.344538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.345821 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.345833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.346605 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.346612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.346830 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.347428 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:03.347451 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:03.349655 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:03.352679 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:03.352688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:03.352705 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:03.353291 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:03.353692 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:03.353697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:03.353700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:03.353953 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:03.354590 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:03.354598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:03.354942 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.355190 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.355721 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":392,"threads_started":1}
I20260808 13:15:03.358184 19526 heartbeater.cc:347] Connected to a master server at 127.31.59.190:33777
I20260808 13:15:03.358467 19526 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:03.359124 19526 heartbeater.cc:511] Master 127.31.59.190:33777 requested a full tablet report, sending...
I20260808 13:15:03.360458 19030 ts_manager.cc:194] Registered new tserver with Master: b5d07336bac04f81adee6896ec5ad594 (127.31.59.131:37299)
I20260808 13:15:03.361233 19030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:60979
I20260808 13:15:03.366561 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:15:03.383425 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.370645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.31.59.190:33777)
0808 13:15:03.370838 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.370844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.371272 (+ 428us) server_negotiation.cc:407] Connection header received
0808 13:15:03.371406 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.371409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.371474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.371550 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:03.373067 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.373899 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.375196 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.375585 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.375672 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.378475 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:03.378480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:03.378483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:03.378515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:03.380868 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.381687 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.381691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.381694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.381818 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.382151 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.382154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.382156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.382456 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:03.382653 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.382954 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.383108 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:15:03.387396 19030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:15:03.394367 19030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:15:03.416878 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.409062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.31.59.131:37299)
0808 13:15:03.409876 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.409881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.409906 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:15:03.409976 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.409981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.410136 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.410336 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:03.410937 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.412503 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.414678 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.415771 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.415887 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.416123 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.416375 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.416543 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":537,"threads_started":1}
I20260808 13:15:03.417142 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.408459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37299 (local address 127.0.0.1:34594)
0808 13:15:03.409062 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.409079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.409191 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.410321 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.410325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.410347 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:03.410701 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.410715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.412645 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.412650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.414496 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.414510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.415948 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.415955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.416139 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.416196 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.416312 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.416502 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":60,"thread_start_us":826,"threads_started":1}
I20260808 13:15:03.417914 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.406163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:36943 (local address 127.0.0.1:46494)
0808 13:15:03.406876 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.406921 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.407090 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.409344 (+ 2254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.409352 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.409380 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:03.409976 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.409989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.412532 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.412538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.414934 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.414948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.416865 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.416873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.417207 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.417242 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.417324 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.417479 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":73,"spinlock_wait_cycles":4352,"thread_start_us":350,"threads_started":1}
I20260808 13:15:03.418431 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.406328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33902)
0808 13:15:03.408351 (+ 2023us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.408369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.408492 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.409498 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.409503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.409529 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:03.409985 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.409995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.411877 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.411882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.414477 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.414492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.417095 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.417103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.417702 (+ 599us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.417740 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.417836 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.418058 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1901,"mutex_wait_us":26,"thread_start_us":1743,"threads_started":1}
I20260808 13:15:03.418809 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.407031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.31.59.129:41795)
0808 13:15:03.407910 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.407919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.408611 (+ 692us) server_negotiation.cc:407] Connection header received
0808 13:15:03.408731 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.408739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.408984 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.409180 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:03.410199 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.411737 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.414935 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.416384 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.416513 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.416738 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.417910 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.418074 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":545,"threads_started":1}
I20260808 13:15:03.420069 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.406337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.31.59.130:36943)
0808 13:15:03.408803 (+ 2466us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.408811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.408835 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:15:03.408928 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.408934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.409119 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.410190 (+ 1071us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:03.410772 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.412380 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.415113 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.416574 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.417083 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.418410 (+ 1327us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.418495 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.419718 (+ 1223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2301,"thread_start_us":2159,"threads_started":1}
I20260808 13:15:03.421034 19457 tablet_service.cc:1467] Processing CreateTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f (DEFAULT_TABLE table=test-table [id=6f73bc3b2e644c68a20bbaebfbc5aed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:03.422115 19164 tablet_service.cc:1467] Processing CreateTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f (DEFAULT_TABLE table=test-table [id=6f73bc3b2e644c68a20bbaebfbc5aed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:03.422669 19457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e0ffe5624c4aa4ae9640b338817d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:03.423180 19164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e0ffe5624c4aa4ae9640b338817d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:03.423493 19305 tablet_service.cc:1467] Processing CreateTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f (DEFAULT_TABLE table=test-table [id=6f73bc3b2e644c68a20bbaebfbc5aed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:03.424780 19305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e0ffe5624c4aa4ae9640b338817d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:03.435981 19547 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Bootstrap starting.
I20260808 13:15:03.439988 19549 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Bootstrap starting.
I20260808 13:15:03.440737 19547 tablet_bootstrap.cc:654] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:03.442239 19547 log.cc:824] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:03.442965 19548 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Bootstrap starting.
I20260808 13:15:03.444749 19549 tablet_bootstrap.cc:654] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:03.445796 19547 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: No bootstrap required, opened a new log
I20260808 13:15:03.445931 19548 tablet_bootstrap.cc:654] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:03.446125 19547 ts_tablet_manager.cc:1397] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 13:15:03.446445 19549 log.cc:824] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:03.447098 19548 log.cc:824] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:03.450533 19548 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: No bootstrap required, opened a new log
I20260808 13:15:03.450779 19548 ts_tablet_manager.cc:1397] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260808 13:15:03.452054 19549 tablet_bootstrap.cc:492] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: No bootstrap required, opened a new log
I20260808 13:15:03.452349 19549 ts_tablet_manager.cc:1397] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260808 13:15:03.457154 19547 raft_consensus.cc:348] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.457653 19547 raft_consensus.cc:374] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:03.457753 19547 raft_consensus.cc:729] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b758e4cc6e7467caf31af6e202c446f, State: Initialized, Role: FOLLOWER
I20260808 13:15:03.458070 19548 raft_consensus.cc:348] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.458479 19548 raft_consensus.cc:374] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:03.458577 19548 raft_consensus.cc:729] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d07336bac04f81adee6896ec5ad594, State: Initialized, Role: FOLLOWER
I20260808 13:15:03.458482 19547 consensus_queue.cc:260] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.459146 19548 consensus_queue.cc:260] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.460553 19547 ts_tablet_manager.cc:1428] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 13:15:03.461517 19526 heartbeater.cc:503] Master 127.31.59.190:33777 was elected leader, sending a full tablet report...
I20260808 13:15:03.462143 19548 ts_tablet_manager.cc:1428] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260808 13:15:03.463820 19549 raft_consensus.cc:348] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.464241 19549 raft_consensus.cc:374] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:03.464347 19549 raft_consensus.cc:729] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31b874e03f440e397bd376bac1bf3af, State: Initialized, Role: FOLLOWER
I20260808 13:15:03.464891 19549 consensus_queue.cc:260] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.466351 19374 heartbeater.cc:503] Master 127.31.59.190:33777 was elected leader, sending a full tablet report...
I20260808 13:15:03.466737 19549 ts_tablet_manager.cc:1428] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260808 13:15:03.492880 19233 tablet.cc:2369] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:15:03.532016 19375 tablet.cc:2369] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:15:03.587001 19527 tablet.cc:2369] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:03.736217 19556 raft_consensus.cc:482] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:15:03.736521 19556 raft_consensus.cc:504] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.738437 19556 leader_election.cc:288] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f31b874e03f440e397bd376bac1bf3af (127.31.59.130:36943), b5d07336bac04f81adee6896ec5ad594 (127.31.59.131:37299)
I20260808 13:15:03.745421 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.739220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:59413 (local address 127.31.59.131:37299)
0808 13:15:03.739446 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.739453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.740069 (+ 616us) server_negotiation.cc:407] Connection header received
0808 13:15:03.740230 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.740234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.740302 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.740383 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:03.741045 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.741851 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.743635 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.744442 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.744569 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.744627 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.745007 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.745153 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:15:03.745472 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.738971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54609 (local address 127.31.59.130:36943)
0808 13:15:03.739216 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.739223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.740025 (+ 802us) server_negotiation.cc:407] Connection header received
0808 13:15:03.740210 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.740215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.740289 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.740357 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:03.740970 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.741751 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.743620 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.744563 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.744719 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.744793 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.745088 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.745217 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 13:15:03.745558 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.738772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37299 (local address 127.31.59.129:59413)
0808 13:15:03.739949 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.739964 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.740102 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.740414 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.740418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.740441 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:03.740890 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.740903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.741981 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.741985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.743488 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.743501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.744650 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.744656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.744856 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.744881 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.744954 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.745070 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":37,"thread_start_us":839,"threads_started":1}
I20260808 13:15:03.745787 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.738669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:36943 (local address 127.31.59.129:54609)
0808 13:15:03.739894 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.739911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:03.740059 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:03.740400 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:03.740405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:03.740428 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:03.740822 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.740835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.741871 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.741876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.743463 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:03.743476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.744789 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.744794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:03.744919 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.744949 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:15:03.745041 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:15:03.745164 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":38,"thread_start_us":794,"threads_started":1}
I20260808 13:15:03.746574 19478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e0ffe5624c4aa4ae9640b338817d8f" candidate_uuid: "5b758e4cc6e7467caf31af6e202c446f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d07336bac04f81adee6896ec5ad594" is_pre_election: true
I20260808 13:15:03.746711 19326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e0ffe5624c4aa4ae9640b338817d8f" candidate_uuid: "5b758e4cc6e7467caf31af6e202c446f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31b874e03f440e397bd376bac1bf3af" is_pre_election: true
I20260808 13:15:03.747072 19478 raft_consensus.cc:2454] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b758e4cc6e7467caf31af6e202c446f in term 0.
I20260808 13:15:03.747186 19326 raft_consensus.cc:2454] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b758e4cc6e7467caf31af6e202c446f in term 0.
I20260808 13:15:03.747754 19114 leader_election.cc:302] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b758e4cc6e7467caf31af6e202c446f, b5d07336bac04f81adee6896ec5ad594; no voters:
I20260808 13:15:03.748158 19556 raft_consensus.cc:2781] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:15:03.748315 19556 raft_consensus.cc:482] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:15:03.748422 19556 raft_consensus.cc:3037] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:03.750649 19556 raft_consensus.cc:504] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.751235 19556 leader_election.cc:288] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [CANDIDATE]: Term 1 election: Requested vote from peers f31b874e03f440e397bd376bac1bf3af (127.31.59.130:36943), b5d07336bac04f81adee6896ec5ad594 (127.31.59.131:37299)
I20260808 13:15:03.751663 19326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e0ffe5624c4aa4ae9640b338817d8f" candidate_uuid: "5b758e4cc6e7467caf31af6e202c446f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31b874e03f440e397bd376bac1bf3af"
I20260808 13:15:03.751737 19478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e0ffe5624c4aa4ae9640b338817d8f" candidate_uuid: "5b758e4cc6e7467caf31af6e202c446f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d07336bac04f81adee6896ec5ad594"
I20260808 13:15:03.751852 19326 raft_consensus.cc:3037] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:03.751935 19478 raft_consensus.cc:3037] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:03.754091 19326 raft_consensus.cc:2454] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b758e4cc6e7467caf31af6e202c446f in term 1.
I20260808 13:15:03.754194 19478 raft_consensus.cc:2454] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b758e4cc6e7467caf31af6e202c446f in term 1.
I20260808 13:15:03.754505 19113 leader_election.cc:302] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b758e4cc6e7467caf31af6e202c446f, f31b874e03f440e397bd376bac1bf3af; no voters:
I20260808 13:15:03.754855 19556 raft_consensus.cc:2781] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:03.755314 19556 raft_consensus.cc:686] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 1 LEADER]: Becoming Leader. State: Replica: 5b758e4cc6e7467caf31af6e202c446f, State: Running, Role: LEADER
I20260808 13:15:03.755791 19556 consensus_queue.cc:237] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } }
I20260808 13:15:03.760571 19030 catalog_manager.cc:5495] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b758e4cc6e7467caf31af6e202c446f (127.31.59.129). New cstate: current_term: 1 leader_uuid: "5b758e4cc6e7467caf31af6e202c446f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b758e4cc6e7467caf31af6e202c446f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 } health_report { overall_health: UNKNOWN } } }
I20260808 13:15:03.786535 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.780295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.31.59.129:41795)
0808 13:15:03.780529 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.780534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.780618 (+ 84us) server_negotiation.cc:407] Connection header received
0808 13:15:03.780814 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.780817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.780899 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.781020 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:03.781470 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.782199 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.784288 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.784690 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.784796 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.785365 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:03.785906 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:03.786057 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.786216 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.786257 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:15:03.794833 19326 raft_consensus.cc:1261] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 1 FOLLOWER]: Refusing update from remote peer 5b758e4cc6e7467caf31af6e202c446f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:03.794833 19478 raft_consensus.cc:1261] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 1 FOLLOWER]: Refusing update from remote peer 5b758e4cc6e7467caf31af6e202c446f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:03.795862 19562 consensus_queue.cc:1035] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5d07336bac04f81adee6896ec5ad594" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:03.796229 19556 consensus_queue.cc:1035] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31b874e03f440e397bd376bac1bf3af" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:03.806087 19566 mvcc.cc:205] Tried to move back new op lower bound from 7316254325933043712 to 7316254325787336704. Current Snapshot: MvccSnapshot[applied={T|T < 7316254325933043712}]
I20260808 13:15:03.825150 19028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:03.825527 19028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:03.829646 19028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca9382158f204c0bbbcd5a2a4bd82652: Sending DeleteTablet for 3 replicas of tablet 37e0ffe5624c4aa4ae9640b338817d8f
I20260808 13:15:03.830840 19305 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:46494
I20260808 13:15:03.830960 19164 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:33902
I20260808 13:15:03.831256 19457 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e0ffe5624c4aa4ae9640b338817d8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:03 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:34594
I20260808 13:15:03.835188 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19096
I20260808 13:15:03.841137 19579 tablet_replica.cc:330] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: stopping tablet replica
I20260808 13:15:03.841655 19579 raft_consensus.cc:2229] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:03.842216 19579 raft_consensus.cc:2258] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:03.842381 19577 tablet_replica.cc:330] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: stopping tablet replica
I20260808 13:15:03.844434 19579 ts_tablet_manager.cc:1905] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:03.844985 19577 raft_consensus.cc:2229] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:03.845643 19577 raft_consensus.cc:2258] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:03.851768 19579 ts_tablet_manager.cc:1918] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:03.851955 19579 log.cc:1196] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/37e0ffe5624c4aa4ae9640b338817d8f
I20260808 13:15:03.852530 19579 ts_tablet_manager.cc:1939] T 37e0ffe5624c4aa4ae9640b338817d8f P b5d07336bac04f81adee6896ec5ad594: Deleting consensus metadata
I20260808 13:15:03.853695 19577 ts_tablet_manager.cc:1905] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:03.854645 19014 catalog_manager.cc:4841] TS b5d07336bac04f81adee6896ec5ad594 (127.31.59.131:37299): tablet 37e0ffe5624c4aa4ae9640b338817d8f (table test-table [id=6f73bc3b2e644c68a20bbaebfbc5aed9]) successfully deleted
I20260808 13:15:03.856690 19578 tablet_replica.cc:330] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: stopping tablet replica
I20260808 13:15:03.857079 19578 raft_consensus.cc:2229] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 1 LEADER]: Raft consensus shutting down.
Aug 08 13:15:03 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:15:03.860034 19578 raft_consensus.cc:2258] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:03.861184 19577 ts_tablet_manager.cc:1918] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:03.861409 19577 log.cc:1196] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/37e0ffe5624c4aa4ae9640b338817d8f
I20260808 13:15:03.862108 19577 ts_tablet_manager.cc:1939] T 37e0ffe5624c4aa4ae9640b338817d8f P f31b874e03f440e397bd376bac1bf3af: Deleting consensus metadata
I20260808 13:15:03.862540 19578 ts_tablet_manager.cc:1905] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:03.864213 19013 catalog_manager.cc:4841] TS f31b874e03f440e397bd376bac1bf3af (127.31.59.130:36943): tablet 37e0ffe5624c4aa4ae9640b338817d8f (table test-table [id=6f73bc3b2e644c68a20bbaebfbc5aed9]) successfully deleted
I20260808 13:15:03.869395 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.836661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.31.59.129:41795)
0808 13:15:03.836870 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.836877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.848069 (+ 11192us) server_negotiation.cc:407] Connection header received
0808 13:15:03.848600 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.848607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.848704 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.848797 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:03.850854 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.852235 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.854401 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.855213 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.855333 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:03.861629 (+ 6296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:03.861643 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:03.861661 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:03.861712 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:03.865923 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.866869 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.866879 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.866882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.867050 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:03.867526 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:03.867532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:03.867539 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:03.868081 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:03.868516 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:03.868760 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 13:15:03.869028 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:15:03.870148 19226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:03.971385 19578 ts_tablet_manager.cc:1918] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:03.971659 19578 log.cc:1196] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/37e0ffe5624c4aa4ae9640b338817d8f
I20260808 13:15:03.972162 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19096
I20260808 13:15:03.972309 19578 ts_tablet_manager.cc:1939] T 37e0ffe5624c4aa4ae9640b338817d8f P 5b758e4cc6e7467caf31af6e202c446f: Deleting consensus metadata
W20260808 13:15:03.992312 19014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:41795 (error 108)
I20260808 13:15:03.992456 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19238
I20260808 13:15:03.994593 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.993853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33940)
0808 13:15:03.994307 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:03.994421 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":246,"threads_started":1}
W20260808 13:15:03.994804 19014 catalog_manager.cc:4584] TS 5b758e4cc6e7467caf31af6e202c446f (127.31.59.129:41795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37e0ffe5624c4aa4ae9640b338817d8f: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Aug 08 13:15:03 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:15:04.007122 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:03.993280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.31.59.130:36943)
0808 13:15:03.993466 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:03.993472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:03.993526 (+ 54us) server_negotiation.cc:407] Connection header received
0808 13:15:03.993691 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:03.993694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:03.993762 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:03.993856 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:03.995398 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.996122 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.997321 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:03.997787 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:03.997890 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:04.001907 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:04.001924 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:04.001941 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:04.001976 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:04.004569 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:04.005223 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:04.005229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:04.005232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:04.005362 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:04.005755 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:04.005759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:04.005762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:04.006167 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:04.006409 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:04.006620 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 13:15:04.006791 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:15:04.007754 19369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:04.048456 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.047966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33950)
0808 13:15:04.048188 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.048292 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 13:15:04.049626 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.049223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33952)
0808 13:15:04.049370 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.049483 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 13:15:04.099399 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19238
I20260808 13:15:04.110288 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.109759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33966)
0808 13:15:04.109989 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.110123 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 13:15:04.111356 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.110969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33978)
0808 13:15:04.111127 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.111203 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:15:04.120046 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19380
Aug 08 13:15:04 dist-test-slave-49h3 krb5kdc[18982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:15:04.135073 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.120797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.31.59.131:37299)
0808 13:15:04.121004 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:04.121008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:04.121031 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:15:04.121201 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:04.121205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:04.121271 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:04.121353 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:04.122835 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:04.123598 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:04.124874 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:04.125285 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:04.125389 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:04.129848 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:04.129866 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:04.129883 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:04.129922 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:04.132510 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:04.133218 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:04.133225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:04.133227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:04.133382 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:04.133740 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:04.133745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:04.133749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:04.134154 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:04.134395 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:04.134621 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 13:15:04.134812 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:15:04.135725 19521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:04.192183 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.191646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33988)
0808 13:15:04.191918 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.192019 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 13:15:04.193339 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.192909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:41795 (local address 127.0.0.1:33994)
0808 13:15:04.193108 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:04.193202 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 13:15:04.225641 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19380
I20260808 13:15:04.244041 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 18996
I20260808 13:15:04.245163 19060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:04.357690 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 18996
2026-08-08T13:15:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4632 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T13:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:04Z Disabled control of system clock
I20260808 13:15:04.451582 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34301
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:34301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:15:04.792654 19590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:04.792935 19590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:04.793066 19590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:04.800313 19590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:04.800417 19590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:04.800479 19590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:04.800557 19590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:04.800616 19590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:04.809926 19590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:34301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:04.811260 19590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:04.813090 19590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:04.824306 19600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:04.824349 19598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:04.824371 19597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:04.825107 19590 server_base.cc:1034] running on GCE node
I20260808 13:15:04.825925 19590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:04.827345 19590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:04.828552 19590 hybrid_clock.cc:648] HybridClock initialized: now 1786194904828494 us; error 55 us; skew 500 ppm
I20260808 13:15:04.828903 19590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:04.831162 19590 webserver.cc:460] Webserver started at http://127.31.59.190:44347/ using document root <none> and password file <none>
I20260808 13:15:04.831892 19590 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:04.831996 19590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:04.832315 19590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:04.835251 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "c9be9b32fb764a18ac66d6262dac900a"
format_stamp: "Formatted at 2026-08-08 13:15:04 on dist-test-slave-49h3"
server_key: "33a4180719ab35039a19d015f5f208d3"
server_key_iv: "04b2524ac8ecf27eaf11427c233f578c"
server_key_version: "encryptionkey@0"
I20260808 13:15:04.836051 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "c9be9b32fb764a18ac66d6262dac900a"
format_stamp: "Formatted at 2026-08-08 13:15:04 on dist-test-slave-49h3"
server_key: "33a4180719ab35039a19d015f5f208d3"
server_key_iv: "04b2524ac8ecf27eaf11427c233f578c"
server_key_version: "encryptionkey@0"
I20260808 13:15:04.841187 19590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 13:15:04.844766 19606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:04.846184 19590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:15:04.846380 19590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "c9be9b32fb764a18ac66d6262dac900a"
format_stamp: "Formatted at 2026-08-08 13:15:04 on dist-test-slave-49h3"
server_key: "33a4180719ab35039a19d015f5f208d3"
server_key_iv: "04b2524ac8ecf27eaf11427c233f578c"
server_key_version: "encryptionkey@0"
I20260808 13:15:04.846596 19590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:04.866464 19590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:04.867394 19590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:04.867693 19590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:04.886304 19590 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:34301
I20260808 13:15:04.886335 19662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:34301 every 8 connection(s)
I20260808 13:15:04.887724 19590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:04.891417 19663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:04.893023 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19590
I20260808 13:15:04.893220 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:04.893832 31982 external_mini_cluster.cc:1428] Setting key 198e322d33811f29b033fa3fdfd822f9
I20260808 13:15:04.903753 19663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: Bootstrap starting.
I20260808 13:15:04.907372 19663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:04.907994 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:04.895877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.31.59.190:34301)
0808 13:15:04.896918 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:04.896929 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:04.896975 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:15:04.898727 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:04.898750 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:04.899195 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:04.899507 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:04.900161 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:04.901581 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:04.903472 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:04.904197 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:04.904350 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:04.905610 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:04.905618 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:04.905631 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:04.906181 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:04.906285 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:04.906312 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:04.906403 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:04.906635 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 13:15:04.906937 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":392,"threads_started":1}
I20260808 13:15:04.908473 19663 log.cc:824] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:04.911481 19663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: No bootstrap required, opened a new log
I20260808 13:15:04.918742 19663 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } }
I20260808 13:15:04.919081 19663 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:04.919162 19663 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9be9b32fb764a18ac66d6262dac900a, State: Initialized, Role: FOLLOWER
I20260808 13:15:04.919641 19663 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } }
I20260808 13:15:04.919869 19663 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:04.919970 19663 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:04.920089 19663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:04.922266 19663 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } }
I20260808 13:15:04.922710 19663 leader_election.cc:302] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9be9b32fb764a18ac66d6262dac900a; no voters:
I20260808 13:15:04.923331 19663 leader_election.cc:288] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:04.923507 19669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:04.924190 19669 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [term 1 LEADER]: Becoming Leader. State: Replica: c9be9b32fb764a18ac66d6262dac900a, State: Running, Role: LEADER
I20260808 13:15:04.924707 19669 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } }
I20260808 13:15:04.925366 19663 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:04.927596 19670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9be9b32fb764a18ac66d6262dac900a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } } }
I20260808 13:15:04.927956 19671 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9be9b32fb764a18ac66d6262dac900a. Latest consensus state: current_term: 1 leader_uuid: "c9be9b32fb764a18ac66d6262dac900a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be9b32fb764a18ac66d6262dac900a" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 34301 } } }
I20260808 13:15:04.928308 19671 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:04.928187 19670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:04.929246 19677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:04.934875 19677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:04.946569 19677 catalog_manager.cc:1347] Generated new cluster ID: f0635557a5bc49d4a6aad5e071976c58
I20260808 13:15:04.946694 19677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:04.956358 19677 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:04.958276 19677 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:04.966704 19677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: Generated new TSK 0
I20260808 13:15:04.967782 19677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:04.972220 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--builtin_ntp_servers=127.31.59.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:15:05.315966 19689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:05.316277 19689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:05.316416 19689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:05.323921 19689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:05.324034 19689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:05.324180 19689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:15:05.333932 19689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:05.335271 19689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:05.337118 19689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:05.350108 19695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:05.350165 19696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:05.350142 19699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:05.351114 19689 server_base.cc:1034] running on GCE node
I20260808 13:15:05.351729 19689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:05.352692 19689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:05.353906 19689 hybrid_clock.cc:648] HybridClock initialized: now 1786194905353843 us; error 55 us; skew 500 ppm
I20260808 13:15:05.354238 19689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:05.356482 19689 webserver.cc:460] Webserver started at http://127.31.59.129:42147/ using document root <none> and password file <none>
I20260808 13:15:05.357159 19689 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:05.357276 19689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:05.357631 19689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:05.360562 19689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "7181dc771f37427ba29c5c93a7c4cfa6"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "d0f33dc8f0cc28227482ccccf6d107b2"
server_key_iv: "a3457b8ab0c023f51e790cd56dd75513"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.361364 19689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "7181dc771f37427ba29c5c93a7c4cfa6"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "d0f33dc8f0cc28227482ccccf6d107b2"
server_key_iv: "a3457b8ab0c023f51e790cd56dd75513"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.366557 19689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:15:05.370036 19705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.371296 19689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:05.371454 19689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "7181dc771f37427ba29c5c93a7c4cfa6"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "d0f33dc8f0cc28227482ccccf6d107b2"
server_key_iv: "a3457b8ab0c023f51e790cd56dd75513"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.371655 19689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:05.402652 19689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:05.403590 19689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:05.403914 19689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:05.404986 19689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:05.406539 19689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:05.406644 19689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.406754 19689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:05.406826 19689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.438937 19689 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:35063
I20260808 13:15:05.439018 19828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:35063 every 8 connection(s)
I20260808 13:15:05.440249 19689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:15:05.444564 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19689
I20260808 13:15:05.444730 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:15:05.445173 31982 external_mini_cluster.cc:1428] Setting key fad917e2dae602085ea8e6e6dcfb2d98
I20260808 13:15:05.448287 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--builtin_ntp_servers=127.31.59.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:15:05.453836 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:05.442805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:38945 (local address 127.31.59.190:34301)
0808 13:15:05.443268 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:05.443275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:05.444525 (+ 1250us) server_negotiation.cc:407] Connection header received
0808 13:15:05.446534 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:05.446539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:05.446641 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:05.446746 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:05.448130 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.448909 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.451065 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.451547 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.451642 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:05.453096 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:05.453100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:05.453104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:05.453122 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:05.453164 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:05.453176 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:05.453254 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:05.453482 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 13:15:05.453563 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":207,"threads_started":1}
I20260808 13:15:05.455155 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:05.443440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34301 (local address 127.31.59.129:38945)
0808 13:15:05.444362 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:05.444409 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:05.446279 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:05.447006 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:05.447015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:05.447116 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:05.447931 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:05.447958 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.449061 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.449067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:05.450939 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:05.450954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.451776 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.451784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:05.452060 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:05.452777 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:05.452780 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:05.452830 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:05.452920 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:05.453263 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:05.453273 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:05.453280 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:15:05.453472 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 13:15:05.453849 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":273,"threads_started":1}
I20260808 13:15:05.456626 19829 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34301
I20260808 13:15:05.456940 19829 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:05.457878 19829 heartbeater.cc:511] Master 127.31.59.190:34301 requested a full tablet report, sending...
I20260808 13:15:05.459995 19625 ts_manager.cc:194] Registered new tserver with Master: 7181dc771f37427ba29c5c93a7c4cfa6 (127.31.59.129:35063)
I20260808 13:15:05.461557 19625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.129:38945
W20260808 13:15:05.786592 19833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:05.786875 19833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:05.787009 19833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:05.794323 19833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:05.794428 19833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:05.794581 19833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:15:05.803857 19833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:05.805145 19833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:05.807018 19833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:05.819732 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:05.819689 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:05.819767 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:05.820562 19833 server_base.cc:1034] running on GCE node
I20260808 13:15:05.821247 19833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:05.822191 19833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:05.823388 19833 hybrid_clock.cc:648] HybridClock initialized: now 1786194905823342 us; error 40 us; skew 500 ppm
I20260808 13:15:05.823685 19833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:05.825897 19833 webserver.cc:460] Webserver started at http://127.31.59.130:44579/ using document root <none> and password file <none>
I20260808 13:15:05.826603 19833 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:05.826697 19833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:05.827003 19833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:05.830075 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "cb54647f3e3e4eea8c4418254ead0da5"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "4a7970ec74ef51435aa8b0d1bdc38c01"
server_key_iv: "1097ec5b680d46b61c223809a556fd97"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.830921 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "cb54647f3e3e4eea8c4418254ead0da5"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "4a7970ec74ef51435aa8b0d1bdc38c01"
server_key_iv: "1097ec5b680d46b61c223809a556fd97"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.836042 19833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:15:05.839512 19847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.840817 19833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:15:05.840971 19833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "cb54647f3e3e4eea8c4418254ead0da5"
format_stamp: "Formatted at 2026-08-08 13:15:05 on dist-test-slave-49h3"
server_key: "4a7970ec74ef51435aa8b0d1bdc38c01"
server_key_iv: "1097ec5b680d46b61c223809a556fd97"
server_key_version: "encryptionkey@0"
I20260808 13:15:05.841167 19833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:05.865813 19833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:05.866739 19833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:05.867048 19833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:05.868140 19833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:05.869755 19833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:05.869858 19833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.869969 19833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:05.870041 19833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:05.906512 19833 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:39097
I20260808 13:15:05.906584 19968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:39097 every 8 connection(s)
I20260808 13:15:05.907802 19833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:15:05.911377 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19833
I20260808 13:15:05.911546 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:15:05.911976 31982 external_mini_cluster.cc:1428] Setting key 60535ac65ec57b6970829afb97e9a62b
I20260808 13:15:05.914882 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--builtin_ntp_servers=127.31.59.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:15:05.920763 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:05.910597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:37093 (local address 127.31.59.190:34301)
0808 13:15:05.910823 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:05.910829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:05.912422 (+ 1593us) server_negotiation.cc:407] Connection header received
0808 13:15:05.914702 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:05.914707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:05.914796 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:05.914878 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:05.915941 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.916641 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.918219 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.918613 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.918730 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:05.919986 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:05.919992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:05.919996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:05.920014 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:05.920047 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:05.920059 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:05.920132 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:05.920343 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 13:15:05.920447 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:15:05.921772 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:05.911215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34301 (local address 127.31.59.130:37093)
0808 13:15:05.912252 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:05.912307 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:05.914449 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:05.915122 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:05.915130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:05.915228 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:05.915781 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:05.915799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.916804 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.916810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:05.918081 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:05.918094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:05.918828 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:05.918836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:05.919053 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:05.919689 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:05.919692 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:05.919730 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:05.919809 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:05.920144 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:05.920157 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:05.920164 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:15:05.920317 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 13:15:05.920696 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":315,"threads_started":1}
I20260808 13:15:05.923091 19969 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34301
I20260808 13:15:05.923671 19969 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:05.925153 19969 heartbeater.cc:511] Master 127.31.59.190:34301 requested a full tablet report, sending...
I20260808 13:15:05.927340 19625 ts_manager.cc:194] Registered new tserver with Master: cb54647f3e3e4eea8c4418254ead0da5 (127.31.59.130:39097)
I20260808 13:15:05.928157 19625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.130:37093
W20260808 13:15:06.255137 19972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:06.255429 19972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:06.255556 19972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:06.263054 19972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:06.263171 19972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:06.263332 19972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:15:06.272486 19972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:06.273873 19972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:06.275707 19972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:06.288384 19977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:06.288309 19980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:06.288332 19978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:06.289379 19972 server_base.cc:1034] running on GCE node
I20260808 13:15:06.289980 19972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:06.290901 19972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:06.292099 19972 hybrid_clock.cc:648] HybridClock initialized: now 1786194906292059 us; error 33 us; skew 500 ppm
I20260808 13:15:06.292377 19972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:06.294550 19972 webserver.cc:460] Webserver started at http://127.31.59.131:36739/ using document root <none> and password file <none>
I20260808 13:15:06.295197 19972 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:06.295284 19972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:06.295562 19972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:06.298475 19972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f"
format_stamp: "Formatted at 2026-08-08 13:15:06 on dist-test-slave-49h3"
server_key: "786e68001e68e64aa31ed9596e82a683"
server_key_iv: "45e76231d8f07934a39f036f6b9e4dea"
server_key_version: "encryptionkey@0"
I20260808 13:15:06.299214 19972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f"
format_stamp: "Formatted at 2026-08-08 13:15:06 on dist-test-slave-49h3"
server_key: "786e68001e68e64aa31ed9596e82a683"
server_key_iv: "45e76231d8f07934a39f036f6b9e4dea"
server_key_version: "encryptionkey@0"
I20260808 13:15:06.304267 19972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 13:15:06.307701 19987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:06.308930 19972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:15:06.309058 19972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f"
format_stamp: "Formatted at 2026-08-08 13:15:06 on dist-test-slave-49h3"
server_key: "786e68001e68e64aa31ed9596e82a683"
server_key_iv: "45e76231d8f07934a39f036f6b9e4dea"
server_key_version: "encryptionkey@0"
I20260808 13:15:06.309229 19972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:06.337956 19972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:06.338806 19972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:06.339102 19972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:06.340108 19972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:06.341629 19972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:06.341725 19972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:06.341835 19972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:06.341903 19972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:06.373636 19972 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:42047
I20260808 13:15:06.373719 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:42047 every 8 connection(s)
I20260808 13:15:06.374953 19972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:15:06.383419 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 19972
I20260808 13:15:06.383644 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:15:06.384212 31982 external_mini_cluster.cc:1428] Setting key 5244422a3442cc608934f37344a88ca9
I20260808 13:15:06.387758 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.377393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:39493 (local address 127.31.59.190:34301)
0808 13:15:06.377643 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.377650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.379010 (+ 1360us) server_negotiation.cc:407] Connection header received
0808 13:15:06.380790 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.380794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.380862 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.380946 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:06.382030 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.382794 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.384945 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.385484 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.385606 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.387071 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:06.387077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:06.387082 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:06.387104 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:06.387143 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:06.387160 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:06.387237 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.387435 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.387509 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 13:15:06.388741 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.377989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:34301 (local address 127.31.59.131:39493)
0808 13:15:06.378861 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.378896 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.380563 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.381151 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.381159 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.381241 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:06.381847 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.381865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.382917 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.382920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.384795 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.384807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.385711 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.385719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.386008 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.386760 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:06.386762 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:06.386801 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:06.386877 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:06.387255 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:06.387266 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:06.387274 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.387423 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.387706 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":282,"threads_started":1}
I20260808 13:15:06.389935 20109 heartbeater.cc:347] Connected to a master server at 127.31.59.190:34301
I20260808 13:15:06.390204 20109 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:06.390904 20109 heartbeater.cc:511] Master 127.31.59.190:34301 requested a full tablet report, sending...
I20260808 13:15:06.392182 19625 ts_manager.cc:194] Registered new tserver with Master: cd3b2b46fe5e4914a5d1aee11b5d8d9f (127.31.59.131:42047)
I20260808 13:15:06.392923 19625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.131:39493
I20260808 13:15:06.399281 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:15:06.408797 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.403352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.31.59.190:34301)
0808 13:15:06.403538 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.403543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.403957 (+ 414us) server_negotiation.cc:407] Connection header received
0808 13:15:06.404089 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.404093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.404160 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.404238 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:06.404675 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.405346 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.406770 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.407164 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.407251 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.408225 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:06.408230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:06.408233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:06.408249 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:06.408280 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:06.408291 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:06.408397 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.408523 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.408568 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:15:06.412312 19625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:15:06.419395 19625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:15:06.453142 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.431157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.31.59.130:39097)
0808 13:15:06.444863 (+ 13706us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.444871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.444898 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:15:06.444996 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.445004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.445253 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.445501 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:06.446548 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.448004 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.450813 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.451986 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.452108 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.452368 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.452635 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.452810 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13560,"thread_start_us":13432,"threads_started":1}
I20260808 13:15:06.453176 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.430988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:39097 (local address 127.0.0.1:56106)
0808 13:15:06.442083 (+ 11095us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.442136 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.442341 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.445819 (+ 3478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.445826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.445876 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:06.446363 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.446379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.448182 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.448188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.450628 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.450643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.452177 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.452184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.452399 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.452450 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.452559 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.452771 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10802,"mutex_wait_us":1,"spinlock_wait_cycles":5504,"thread_start_us":12290,"threads_started":1}
I20260808 13:15:06.453859 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.431819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.31.59.131:42047)
0808 13:15:06.432463 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.432473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.444892 (+ 12419us) server_negotiation.cc:407] Connection header received
0808 13:15:06.444997 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.445004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.445253 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.445467 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:06.447067 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.448572 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.450596 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.452035 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.452686 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.452938 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.453019 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.453184 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":328,"threads_started":1}
I20260808 13:15:06.453969 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.431228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:42047 (local address 127.0.0.1:46976)
0808 13:15:06.443055 (+ 11827us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.443073 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.443209 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.445833 (+ 2624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.445837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.445885 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:06.446831 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.446846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.448713 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.448718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.450410 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.450448 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.452272 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.452282 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.452480 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.452519 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.452585 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.453605 (+ 1020us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11683,"mutex_wait_us":1377,"spinlock_wait_cycles":11904,"thread_start_us":10388,"threads_started":1}
I20260808 13:15:06.456403 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.446375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:35063 (local address 127.0.0.1:46794)
0808 13:15:06.447389 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.447404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.447512 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.448417 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.448420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.448440 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:06.448892 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.448908 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.451604 (+ 2696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.451610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.453659 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.453675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.455720 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.455725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.455889 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.455920 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.456003 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.456127 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":747,"threads_started":1}
I20260808 13:15:06.457855 20038 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9a3f885d4f4443b07212290e59c799 (DEFAULT_TABLE table=test-table [id=60743ffbbfdb43fbb3b1897d282863f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:06.462644 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.442743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.31.59.129:35063)
0808 13:15:06.446412 (+ 3669us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.446420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.447640 (+ 1220us) server_negotiation.cc:407] Connection header received
0808 13:15:06.447745 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.447752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.447971 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.448162 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:06.449075 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.451418 (+ 2343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.453890 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.455204 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.455329 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.455577 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.456716 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.456880 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3478,"thread_start_us":3335,"threads_started":1}
I20260808 13:15:06.462963 19899 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9a3f885d4f4443b07212290e59c799 (DEFAULT_TABLE table=test-table [id=60743ffbbfdb43fbb3b1897d282863f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:06.464706 19899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9a3f885d4f4443b07212290e59c799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:06.464689 19829 heartbeater.cc:503] Master 127.31.59.190:34301 was elected leader, sending a full tablet report...
I20260808 13:15:06.467370 20038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9a3f885d4f4443b07212290e59c799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:06.467000 19758 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9a3f885d4f4443b07212290e59c799 (DEFAULT_TABLE table=test-table [id=60743ffbbfdb43fbb3b1897d282863f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:06.468396 19758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9a3f885d4f4443b07212290e59c799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:06.484053 20127 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Bootstrap starting.
I20260808 13:15:06.489413 20128 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Bootstrap starting.
I20260808 13:15:06.490017 20127 tablet_bootstrap.cc:654] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:06.490628 20129 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Bootstrap starting.
I20260808 13:15:06.491402 20127 log.cc:824] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:06.494974 20128 tablet_bootstrap.cc:654] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:06.496331 20128 log.cc:824] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:06.500028 20127 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: No bootstrap required, opened a new log
I20260808 13:15:06.500180 20128 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: No bootstrap required, opened a new log
I20260808 13:15:06.500398 20127 ts_tablet_manager.cc:1397] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260808 13:15:06.500456 20128 ts_tablet_manager.cc:1397] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260808 13:15:06.501437 20129 tablet_bootstrap.cc:654] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:06.502933 20129 log.cc:824] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:06.511337 20128 raft_consensus.cc:348] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.511336 20127 raft_consensus.cc:348] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.511837 20127 raft_consensus.cc:374] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:06.511999 20128 raft_consensus.cc:374] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:06.511983 20127 raft_consensus.cc:729] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb54647f3e3e4eea8c4418254ead0da5, State: Initialized, Role: FOLLOWER
I20260808 13:15:06.512169 20128 raft_consensus.cc:729] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7181dc771f37427ba29c5c93a7c4cfa6, State: Initialized, Role: FOLLOWER
I20260808 13:15:06.512918 20128 consensus_queue.cc:260] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.512895 20127 consensus_queue.cc:260] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.514366 19969 heartbeater.cc:503] Master 127.31.59.190:34301 was elected leader, sending a full tablet report...
I20260808 13:15:06.516438 20129 tablet_bootstrap.cc:492] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: No bootstrap required, opened a new log
I20260808 13:15:06.516477 20127 ts_tablet_manager.cc:1428] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260808 13:15:06.516757 20129 ts_tablet_manager.cc:1397] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.004s
I20260808 13:15:06.517086 20128 ts_tablet_manager.cc:1428] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260808 13:15:06.528182 20129 raft_consensus.cc:348] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.528714 20129 raft_consensus.cc:374] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:06.528824 20129 raft_consensus.cc:729] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd3b2b46fe5e4914a5d1aee11b5d8d9f, State: Initialized, Role: FOLLOWER
I20260808 13:15:06.529548 20129 consensus_queue.cc:260] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.530815 20109 heartbeater.cc:503] Master 127.31.59.190:34301 was elected leader, sending a full tablet report...
I20260808 13:15:06.531562 20129 ts_tablet_manager.cc:1428] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
W20260808 13:15:06.626677 20110 tablet.cc:2369] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:15:06.659907 19970 tablet.cc:2369] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:06.685014 20135 raft_consensus.cc:482] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:15:06.685333 20135 raft_consensus.cc:504] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.687160 20135 leader_election.cc:288] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb54647f3e3e4eea8c4418254ead0da5 (127.31.59.130:39097), 7181dc771f37427ba29c5c93a7c4cfa6 (127.31.59.129:35063)
W20260808 13:15:06.692764 19830 tablet.cc:2369] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:06.695297 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.687477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:35063 (local address 127.31.59.131:57495)
0808 13:15:06.687911 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.687930 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.688058 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.688866 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.688870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.688894 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:06.689090 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.689100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.690549 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.690557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.692876 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.692885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.694576 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.694581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.694761 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.694784 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.694860 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.694991 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":168}
I20260808 13:15:06.695284 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.687763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:54213 (local address 127.31.59.130:39097)
0808 13:15:06.687985 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.687991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.688608 (+ 617us) server_negotiation.cc:407] Connection header received
0808 13:15:06.688766 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.688771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.688862 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.688950 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:06.689645 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.690315 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.693059 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.694278 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.694425 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.694495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.694893 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.695018 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:15:06.695399 20137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.687576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:39097 (local address 127.31.59.131:54213)
0808 13:15:06.688484 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:06.688500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:06.688637 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:06.688973 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:06.688976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:06.688995 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:06.689433 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.689480 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.690444 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.690448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.692854 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:06.692871 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.694489 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.694496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:06.694696 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.694753 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 13:15:06.694844 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 13:15:06.694978 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":114,"thread_start_us":588,"threads_started":1}
I20260808 13:15:06.695926 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.688293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:57495 (local address 127.31.59.129:35063)
0808 13:15:06.688515 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.688523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.688543 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:15:06.688621 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.688626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.688712 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.688798 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:06.689269 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.690405 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.693058 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.694301 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.694430 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.694495 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.695532 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.695691 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 13:15:06.696780 19779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9a3f885d4f4443b07212290e59c799" candidate_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" is_pre_election: true
I20260808 13:15:06.696913 19920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9a3f885d4f4443b07212290e59c799" candidate_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb54647f3e3e4eea8c4418254ead0da5" is_pre_election: true
I20260808 13:15:06.697314 19779 raft_consensus.cc:2454] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd3b2b46fe5e4914a5d1aee11b5d8d9f in term 0.
I20260808 13:15:06.697521 19920 raft_consensus.cc:2454] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd3b2b46fe5e4914a5d1aee11b5d8d9f in term 0.
I20260808 13:15:06.698074 19989 leader_election.cc:302] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7181dc771f37427ba29c5c93a7c4cfa6, cd3b2b46fe5e4914a5d1aee11b5d8d9f; no voters:
I20260808 13:15:06.698707 20135 raft_consensus.cc:2781] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:15:06.698865 20135 raft_consensus.cc:482] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:15:06.698974 20135 raft_consensus.cc:3037] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:06.701416 20135 raft_consensus.cc:504] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.702051 20135 leader_election.cc:288] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [CANDIDATE]: Term 1 election: Requested vote from peers cb54647f3e3e4eea8c4418254ead0da5 (127.31.59.130:39097), 7181dc771f37427ba29c5c93a7c4cfa6 (127.31.59.129:35063)
I20260808 13:15:06.702466 19920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9a3f885d4f4443b07212290e59c799" candidate_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb54647f3e3e4eea8c4418254ead0da5"
I20260808 13:15:06.702595 19779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9a3f885d4f4443b07212290e59c799" candidate_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7181dc771f37427ba29c5c93a7c4cfa6"
I20260808 13:15:06.702694 19920 raft_consensus.cc:3037] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:06.702798 19779 raft_consensus.cc:3037] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:06.705951 19779 raft_consensus.cc:2454] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd3b2b46fe5e4914a5d1aee11b5d8d9f in term 1.
I20260808 13:15:06.705958 19920 raft_consensus.cc:2454] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd3b2b46fe5e4914a5d1aee11b5d8d9f in term 1.
I20260808 13:15:06.706370 19991 leader_election.cc:302] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb54647f3e3e4eea8c4418254ead0da5, cd3b2b46fe5e4914a5d1aee11b5d8d9f; no voters:
I20260808 13:15:06.706684 20135 raft_consensus.cc:2781] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:06.707178 20135 raft_consensus.cc:686] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 1 LEADER]: Becoming Leader. State: Replica: cd3b2b46fe5e4914a5d1aee11b5d8d9f, State: Running, Role: LEADER
I20260808 13:15:06.707808 20135 consensus_queue.cc:237] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } }
I20260808 13:15:06.713055 19624 catalog_manager.cc:5495] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to cd3b2b46fe5e4914a5d1aee11b5d8d9f (127.31.59.131). New cstate: current_term: 1 leader_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd3b2b46fe5e4914a5d1aee11b5d8d9f" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 } health_report { overall_health: UNKNOWN } } }
I20260808 13:15:06.768707 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.763629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.31.59.131:42047)
0808 13:15:06.763827 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.763832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.763849 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:15:06.763935 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.763938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.764014 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.764133 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:06.764460 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.765169 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.766563 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.766970 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.767060 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.767606 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:06.768106 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:06.768251 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.768404 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.768442 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:15:06.776960 19920 raft_consensus.cc:1261] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 1 FOLLOWER]: Refusing update from remote peer cd3b2b46fe5e4914a5d1aee11b5d8d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:06.776961 19779 raft_consensus.cc:1261] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 1 FOLLOWER]: Refusing update from remote peer cd3b2b46fe5e4914a5d1aee11b5d8d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:06.777899 20138 consensus_queue.cc:1035] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7181dc771f37427ba29c5c93a7c4cfa6" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:06.778241 20135 consensus_queue.cc:1035] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb54647f3e3e4eea8c4418254ead0da5" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 39097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:06.790408 20142 mvcc.cc:205] Tried to move back new op lower bound from 7316254338148057088 to 7316254337879965696. Current Snapshot: MvccSnapshot[applied={T|T < 7316254338148057088}]
I20260808 13:15:06.801731 19625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:06.802043 19625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:06.805933 19625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9be9b32fb764a18ac66d6262dac900a: Sending DeleteTablet for 3 replicas of tablet 9a9a3f885d4f4443b07212290e59c799
I20260808 13:15:06.807121 19899 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9a3f885d4f4443b07212290e59c799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:06 UTC) from {username='slave'} at 127.0.0.1:56106
I20260808 13:15:06.807662 19758 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9a3f885d4f4443b07212290e59c799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:06 UTC) from {username='slave'} at 127.0.0.1:46794
I20260808 13:15:06.807818 20038 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9a3f885d4f4443b07212290e59c799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:06 UTC) from {username='slave'} at 127.0.0.1:46976
I20260808 13:15:06.808215 20152 tablet_replica.cc:330] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: stopping tablet replica
I20260808 13:15:06.808636 20152 raft_consensus.cc:2229] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:06.809157 20152 raft_consensus.cc:2258] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:06.809327 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19689
I20260808 13:15:06.812255 20152 ts_tablet_manager.cc:1905] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:06.820021 20152 ts_tablet_manager.cc:1918] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:06.820286 20152 log.cc:1196] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/9a9a3f885d4f4443b07212290e59c799
I20260808 13:15:06.820892 20152 ts_tablet_manager.cc:1939] T 9a9a3f885d4f4443b07212290e59c799 P cb54647f3e3e4eea8c4418254ead0da5: Deleting consensus metadata
I20260808 13:15:06.823177 19611 catalog_manager.cc:4841] TS cb54647f3e3e4eea8c4418254ead0da5 (127.31.59.130:39097): tablet 9a9a3f885d4f4443b07212290e59c799 (table test-table [id=60743ffbbfdb43fbb3b1897d282863f5]) successfully deleted
I20260808 13:15:06.823498 20153 tablet_replica.cc:330] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: stopping tablet replica
I20260808 13:15:06.824062 20153 raft_consensus.cc:2229] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:06.824606 20153 raft_consensus.cc:2258] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:06.827224 20153 ts_tablet_manager.cc:1905] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:06.830989 20154 tablet_replica.cc:330] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: stopping tablet replica
I20260808 13:15:06.831516 20154 raft_consensus.cc:2229] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:15:06.832278 20154 raft_consensus.cc:2258] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:06.834276 20153 ts_tablet_manager.cc:1918] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:06.834386 20154 ts_tablet_manager.cc:1905] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:06.834504 20153 log.cc:1196] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/9a9a3f885d4f4443b07212290e59c799
I20260808 13:15:06.835095 20153 ts_tablet_manager.cc:1939] T 9a9a3f885d4f4443b07212290e59c799 P 7181dc771f37427ba29c5c93a7c4cfa6: Deleting consensus metadata
I20260808 13:15:06.837301 19609 catalog_manager.cc:4841] TS 7181dc771f37427ba29c5c93a7c4cfa6 (127.31.59.129:35063): tablet 9a9a3f885d4f4443b07212290e59c799 (table test-table [id=60743ffbbfdb43fbb3b1897d282863f5]) successfully deleted
I20260808 13:15:06.837749 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.812768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.31.59.129:35063)
0808 13:15:06.812952 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.812958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.825500 (+ 12542us) server_negotiation.cc:407] Connection header received
0808 13:15:06.829626 (+ 4126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.829633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.829776 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.829863 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:06.830727 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.831819 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.833878 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.834478 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.834575 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.836051 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:06.836063 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:06.836080 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:06.836819 (+ 739us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:06.836882 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:06.836908 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:06.836998 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.837233 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.837341 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:15:06.838771 19824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:06.842301 20154 ts_tablet_manager.cc:1918] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:06.842446 20154 log.cc:1196] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/9a9a3f885d4f4443b07212290e59c799
I20260808 13:15:06.842888 20154 ts_tablet_manager.cc:1939] T 9a9a3f885d4f4443b07212290e59c799 P cd3b2b46fe5e4914a5d1aee11b5d8d9f: Deleting consensus metadata
I20260808 13:15:06.844249 19609 catalog_manager.cc:4841] TS cd3b2b46fe5e4914a5d1aee11b5d8d9f (127.31.59.131:42047): tablet 9a9a3f885d4f4443b07212290e59c799 (table test-table [id=60743ffbbfdb43fbb3b1897d282863f5]) successfully deleted
I20260808 13:15:06.946497 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19689
I20260808 13:15:06.966146 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19833
I20260808 13:15:06.973675 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:06.966995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.31.59.130:39097)
0808 13:15:06.967237 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:06.967244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:06.967284 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:15:06.967354 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:06.967360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:06.967469 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:06.967585 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:06.967893 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.968991 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.970326 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:06.970955 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:06.971083 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:06.972051 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:06.972063 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:06.972082 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:06.972862 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:06.972926 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:06.972952 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:06.973071 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:06.973130 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 13:15:06.973242 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:15:06.974417 19964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:07.064262 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19833
I20260808 13:15:07.084110 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19972
I20260808 13:15:07.092026 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:07.084926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.31.59.131:42047)
0808 13:15:07.085142 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:07.085148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:07.085171 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:15:07.085270 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:07.085276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:07.085388 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:07.085541 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:07.085889 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:07.087046 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:07.088354 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:07.089022 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:07.089191 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:07.090335 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:07.090348 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:07.090368 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:07.091230 (+ 862us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:07.091295 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:07.091323 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:07.091470 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:07.091526 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 13:15:07.091627 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:15:07.092859 20104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:07.214747 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19972
I20260808 13:15:07.236075 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 19590
I20260808 13:15:07.237207 19658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:07.347808 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 19590
2026-08-08T13:15:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2987 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:07 dist-test-slave-49h3 krb5kdc[20162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:07 dist-test-slave-49h3 krb5kdc[20162](info): set up 2 sockets
Aug 08 13:15:07 dist-test-slave-49h3 krb5kdc[20162](info): commencing operation
krb5kdc: starting...
W20260808 13:15:08.428455 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:08 dist-test-slave-49h3 krb5kdc[20162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:08.736681 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44777
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:44777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:09.072815 20176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:09.073140 20176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:09.073294 20176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:09.080698 20176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:09.080824 20176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:09.080909 20176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:09.080988 20176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:09.081049 20176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:09.091151 20176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:44777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:09.092518 20176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:09.093842 20176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:09.105317 20184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:09.105381 20182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:09.105289 20181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:09.106554 20176 server_base.cc:1034] running on GCE node
I20260808 13:15:09.107403 20176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:09.108839 20176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:09.110031 20176 hybrid_clock.cc:648] HybridClock initialized: now 1786194909109987 us; error 29 us; skew 500 ppm
I20260808 13:15:09.110385 20176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:09 dist-test-slave-49h3 krb5kdc[20162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:09.117568 20176 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:09.119140 20176 webserver.cc:460] Webserver started at http://127.31.59.190:44049/ using document root <none> and password file <none>
I20260808 13:15:09.119812 20176 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:09.119927 20176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:09.120223 20176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:09.123261 20176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "277a8e0bbc24418a8a4959d61b9c9587"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "e540edeb483761268ea9af206a5fb996"
server_key_iv: "55f1aaba1d5c59524a99ccc1c5918a66"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.123997 20176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "277a8e0bbc24418a8a4959d61b9c9587"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "e540edeb483761268ea9af206a5fb996"
server_key_iv: "55f1aaba1d5c59524a99ccc1c5918a66"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.129230 20176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:15:09.132735 20192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:09.134130 20176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:15:09.134297 20176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "277a8e0bbc24418a8a4959d61b9c9587"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "e540edeb483761268ea9af206a5fb996"
server_key_iv: "55f1aaba1d5c59524a99ccc1c5918a66"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.134482 20176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:09.148998 20176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:09.152885 20176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:09.153158 20176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:09.170713 20176 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:44777
I20260808 13:15:09.170747 20248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:44777 every 8 connection(s)
I20260808 13:15:09.172081 20176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:09.175786 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:09.178347 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20176
I20260808 13:15:09.178520 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:09.178966 31982 external_mini_cluster.cc:1428] Setting key cf6ac7c1621d4b0ca483850a407593bc
I20260808 13:15:09.188541 20249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: Bootstrap starting.
I20260808 13:15:09.192059 20249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:09.193014 20249 log.cc:824] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:15:09 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:09.196072 20249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: No bootstrap required, opened a new log
I20260808 13:15:09.202121 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:09.180748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.31.59.190:44777)
0808 13:15:09.181604 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:09.181619 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:09.181669 (+ 50us) server_negotiation.cc:407] Connection header received
0808 13:15:09.183995 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:09.184023 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:09.184521 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:09.184957 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:09.186293 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.188192 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.189517 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.190578 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.190756 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:09.194970 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:09.194977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:09.194991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:09.195024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:09.198085 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:09.198880 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:09.198887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:09.198895 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:09.199066 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:09.199450 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:09.199453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:09.199455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:09.199976 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:09.200201 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:09.200666 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 13:15:09.201005 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":350,"threads_started":1}
I20260808 13:15:09.205050 20249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } }
I20260808 13:15:09.205418 20249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:09.205544 20249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277a8e0bbc24418a8a4959d61b9c9587, State: Initialized, Role: FOLLOWER
I20260808 13:15:09.206080 20249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } }
I20260808 13:15:09.206302 20249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:09.206430 20249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:09.206569 20249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:09.208840 20249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } }
I20260808 13:15:09.209295 20249 leader_election.cc:302] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 277a8e0bbc24418a8a4959d61b9c9587; no voters:
I20260808 13:15:09.209854 20249 leader_election.cc:288] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:09.209966 20255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:09.210942 20255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [term 1 LEADER]: Becoming Leader. State: Replica: 277a8e0bbc24418a8a4959d61b9c9587, State: Running, Role: LEADER
I20260808 13:15:09.211493 20255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } }
I20260808 13:15:09.211953 20249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:09.214172 20256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "277a8e0bbc24418a8a4959d61b9c9587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } } }
I20260808 13:15:09.214581 20256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:09.214457 20257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277a8e0bbc24418a8a4959d61b9c9587. Latest consensus state: current_term: 1 leader_uuid: "277a8e0bbc24418a8a4959d61b9c9587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277a8e0bbc24418a8a4959d61b9c9587" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 44777 } } }
I20260808 13:15:09.215934 20257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:09.217515 20264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:09.222076 20264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:09.230185 20264 catalog_manager.cc:1347] Generated new cluster ID: e0776ba455fc410497fd07b0f8ab371d
I20260808 13:15:09.230306 20264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:09.240599 20264 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:09.242498 20264 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:09.251792 20264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: Generated new TSK 0
I20260808 13:15:09.252599 20264 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
Entry for principal kudu/127.31.59.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab.
I20260808 13:15:09.330904 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:09.666222 20276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:09.666505 20276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:09.666652 20276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:09.673884 20276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:09.673995 20276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:09.674145 20276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:15:09.683724 20276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.129.keytab
--principal=kudu/127.31.59.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:09.684998 20276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:09.686319 20276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:09.698822 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:09.698822 20284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:09.698863 20282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:09.699714 20276 server_base.cc:1034] running on GCE node
I20260808 13:15:09.700297 20276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:09.701248 20276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:09.702535 20276 hybrid_clock.cc:648] HybridClock initialized: now 1786194909702474 us; error 51 us; skew 500 ppm
I20260808 13:15:09.702837 20276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:09 dist-test-slave-49h3 krb5kdc[20162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:09.705915 20276 init.cc:377] Logged in from keytab as kudu/127.31.59.129@KRBTEST.COM (short username kudu)
I20260808 13:15:09.707350 20276 webserver.cc:460] Webserver started at http://127.31.59.129:34897/ using document root <none> and password file <none>
I20260808 13:15:09.708002 20276 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:09.708123 20276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:09.708412 20276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:09.711252 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "89778dd6d9c94595b46cbdddbad5433b"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "89f3c352f2c95d6e93fd32167de5a7d1"
server_key_iv: "ef431c334bbd153181766e7002dda8b5"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.711984 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "89778dd6d9c94595b46cbdddbad5433b"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "89f3c352f2c95d6e93fd32167de5a7d1"
server_key_iv: "ef431c334bbd153181766e7002dda8b5"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.717634 20276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:15:09.721102 20292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:09.722434 20276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:15:09.722587 20276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "89778dd6d9c94595b46cbdddbad5433b"
format_stamp: "Formatted at 2026-08-08 13:15:09 on dist-test-slave-49h3"
server_key: "89f3c352f2c95d6e93fd32167de5a7d1"
server_key_iv: "ef431c334bbd153181766e7002dda8b5"
server_key_version: "encryptionkey@0"
I20260808 13:15:09.722779 20276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:09.739723 20276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:09.743650 20276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:09.743920 20276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:09.744896 20276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:09.746313 20276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:09.746385 20276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:09.746507 20276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:09.746557 20276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:09.777917 20276 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:40501
I20260808 13:15:09.777961 20417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:40501 every 8 connection(s)
I20260808 13:15:09.779191 20276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:15:09.781805 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20276
I20260808 13:15:09.781988 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:15:09.782526 31982 external_mini_cluster.cc:1428] Setting key a3d9e978d8e37744b9d7183c57cf8dfb
Aug 08 13:15:09 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.129@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:09.800063 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:09.781597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:54855 (local address 127.31.59.190:44777)
0808 13:15:09.782071 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:09.782077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:09.783183 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 13:15:09.785397 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:09.785402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:09.785509 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:09.785623 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:09.787031 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.787944 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.789424 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.789970 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.790084 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:09.793977 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:09.793983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:09.793987 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:09.794023 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:09.796760 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:09.797583 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:09.797587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:09.797590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:09.797717 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:09.798159 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:09.798162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:09.798166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:09.798479 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:09.798615 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:09.799448 (+ 833us) server_negotiation.cc:299] Negotiation successful
0808 13:15:09.799681 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":243,"threads_started":1}
I20260808 13:15:09.801208 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:09.782224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44777 (local address 127.31.59.129:54855)
0808 13:15:09.783012 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:09.783050 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:09.785129 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:09.785807 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:09.785815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:09.786327 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:09.786865 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:09.786882 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.788081 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.788085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:09.789286 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:09.789297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:09.790176 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:09.790182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:09.790430 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:09.791092 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:09.791116 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:09.793755 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:09.796906 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:09.796914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:09.796930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:09.797489 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:09.797814 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:09.797818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:09.797821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:09.798058 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:09.798612 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:09.798621 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:09.799052 (+ 431us) client_negotiation.cc:769] Sending connection context
0808 13:15:09.799315 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 13:15:09.799865 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":334,"threads_started":1}
I20260808 13:15:09.802799 20418 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44777
I20260808 13:15:09.803115 20418 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:09.804095 20418 heartbeater.cc:511] Master 127.31.59.190:44777 requested a full tablet report, sending...
I20260808 13:15:09.806559 20211 ts_manager.cc:194] Registered new tserver with Master: 89778dd6d9c94595b46cbdddbad5433b (127.31.59.129:40501)
I20260808 13:15:09.808528 20211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.129@KRBTEST.COM'} at 127.31.59.129:54855
WARNING: no policy specified for kudu/127.31.59.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
Entry for principal kudu/127.31.59.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab.
I20260808 13:15:09.855922 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:10.206602 20424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:10.206885 20424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:10.207022 20424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:10.214551 20424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:10.214663 20424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:10.214820 20424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:15:10.224804 20424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.130.keytab
--principal=kudu/127.31.59.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:10.226212 20424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:10.227617 20424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:10.240649 20429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:10.240700 20432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:10.240731 20430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:10.241510 20424 server_base.cc:1034] running on GCE node
I20260808 13:15:10.242118 20424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:10.242990 20424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:10.244199 20424 hybrid_clock.cc:648] HybridClock initialized: now 1786194910244143 us; error 45 us; skew 500 ppm
I20260808 13:15:10.244491 20424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:10 dist-test-slave-49h3 krb5kdc[20162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:10.247563 20424 init.cc:377] Logged in from keytab as kudu/127.31.59.130@KRBTEST.COM (short username kudu)
I20260808 13:15:10.248898 20424 webserver.cc:460] Webserver started at http://127.31.59.130:43311/ using document root <none> and password file <none>
I20260808 13:15:10.249594 20424 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:10.249730 20424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:10.250033 20424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:10.252979 20424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "cb9f385053dd42a581975a1c76082d18"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "897b976762035db17eb8de34098012c7"
server_key_iv: "a36814d68a16402766a335d8c7a5c52c"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.253830 20424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "cb9f385053dd42a581975a1c76082d18"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "897b976762035db17eb8de34098012c7"
server_key_iv: "a36814d68a16402766a335d8c7a5c52c"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.258932 20424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 13:15:10.262286 20439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.263532 20424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:10.263679 20424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "cb9f385053dd42a581975a1c76082d18"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "897b976762035db17eb8de34098012c7"
server_key_iv: "a36814d68a16402766a335d8c7a5c52c"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.263866 20424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:10.281802 20424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:10.286116 20424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:10.286397 20424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:10.287369 20424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:10.288734 20424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:10.288808 20424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.288926 20424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:10.288975 20424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.319283 20424 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:34557
I20260808 13:15:10.319337 20559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:34557 every 8 connection(s)
I20260808 13:15:10.320554 20424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:15:10.327921 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20424
I20260808 13:15:10.328114 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:15:10.328578 31982 external_mini_cluster.cc:1428] Setting key a351bd4d4829779b5492f41e23aa38ed
Aug 08 13:15:10 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.130@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:10.340476 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.323201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:57147 (local address 127.31.59.190:44777)
0808 13:15:10.323645 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.323650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.324444 (+ 794us) server_negotiation.cc:407] Connection header received
0808 13:15:10.326369 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.326375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.326457 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.326552 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:10.328019 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.328828 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.330937 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.331373 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.331481 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.334875 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:10.334881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:10.334884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:10.334915 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:10.337377 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.338170 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.338176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.338180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.338315 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.338735 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.338738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.338740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.339038 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:10.339175 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.339869 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.340087 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":217,"threads_started":1}
I20260808 13:15:10.341388 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.323605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44777 (local address 127.31.59.130:57147)
0808 13:15:10.324291 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:10.324330 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:10.326091 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:10.326801 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:10.326809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:10.327300 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:10.327853 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.327870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.328978 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.328982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.330803 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.330816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.331595 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.331603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.331833 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.332484 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:10.332510 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:10.334659 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:10.337523 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:10.337532 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:10.337549 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:10.338052 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:10.338424 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:10.338429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:10.338433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:10.338625 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:10.339159 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:10.339167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:10.339509 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 13:15:10.339757 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 13:15:10.340276 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":267,"threads_started":1}
I20260808 13:15:10.342823 20560 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44777
I20260808 13:15:10.343099 20560 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:10.343832 20560 heartbeater.cc:511] Master 127.31.59.190:44777 requested a full tablet report, sending...
I20260808 13:15:10.345218 20211 ts_manager.cc:194] Registered new tserver with Master: cb9f385053dd42a581975a1c76082d18 (127.31.59.130:34557)
I20260808 13:15:10.346066 20211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.130@KRBTEST.COM'} at 127.31.59.130:57147
WARNING: no policy specified for kudu/127.31.59.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
Entry for principal kudu/127.31.59.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab.
I20260808 13:15:10.396338 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:10.729712 20566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:10.730005 20566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:10.730140 20566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:10.737339 20566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:10.737478 20566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:10.737632 20566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:15:10.747416 20566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.131.keytab
--principal=kudu/127.31.59.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:10.748647 20566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:10.749903 20566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:10.762411 20571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:10.762508 20574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:10.762619 20572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:10.763523 20566 server_base.cc:1034] running on GCE node
I20260808 13:15:10.764148 20566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:10.765039 20566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:10.766235 20566 hybrid_clock.cc:648] HybridClock initialized: now 1786194910766188 us; error 35 us; skew 500 ppm
I20260808 13:15:10.766531 20566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:10 dist-test-slave-49h3 krb5kdc[20162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:10.769552 20566 init.cc:377] Logged in from keytab as kudu/127.31.59.131@KRBTEST.COM (short username kudu)
I20260808 13:15:10.770967 20566 webserver.cc:460] Webserver started at http://127.31.59.131:44283/ using document root <none> and password file <none>
I20260808 13:15:10.771651 20566 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:10.771785 20566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:10.772079 20566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:10.775097 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "86bb5a3978da03f2b203165f3038666e"
server_key_iv: "b2263c4668732eb10649ea42e6feeca6"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.775895 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "86bb5a3978da03f2b203165f3038666e"
server_key_iv: "b2263c4668732eb10649ea42e6feeca6"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.782434 20566 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260808 13:15:10.786235 20581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.787487 20566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:15:10.787638 20566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae"
format_stamp: "Formatted at 2026-08-08 13:15:10 on dist-test-slave-49h3"
server_key: "86bb5a3978da03f2b203165f3038666e"
server_key_iv: "b2263c4668732eb10649ea42e6feeca6"
server_key_version: "encryptionkey@0"
I20260808 13:15:10.787823 20566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:10.802413 20566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:10.806321 20566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:10.806596 20566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:10.807603 20566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:10.808931 20566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:10.809010 20566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.809128 20566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:10.809186 20566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:10.812003 20418 heartbeater.cc:503] Master 127.31.59.190:44777 was elected leader, sending a full tablet report...
I20260808 13:15:10.850904 20566 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:42057
I20260808 13:15:10.850943 20699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:42057 every 8 connection(s)
I20260808 13:15:10.852201 20566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:15:10.858071 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20566
I20260808 13:15:10.858268 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:15:10.858718 31982 external_mini_cluster.cc:1428] Setting key ac91701352f029d898293c751a124c44
Aug 08 13:15:10 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.131@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:10.876374 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.855160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:49889 (local address 127.31.59.190:44777)
0808 13:15:10.855620 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.855624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.856611 (+ 987us) server_negotiation.cc:407] Connection header received
0808 13:15:10.859305 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.859310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.859401 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.859519 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:10.862003 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.862796 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.865105 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.865552 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.865646 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.870190 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:10.870195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:10.870198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:10.870243 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:10.873083 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.874013 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.874019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.874022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.874166 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.874604 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.874610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.874614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.874899 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:10.875039 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.875766 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.875989 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260808 13:15:10.877467 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.855530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:44777 (local address 127.31.59.131:49889)
0808 13:15:10.856401 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:10.856457 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:10.858970 (+ 2513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:10.860132 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:10.860145 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:10.860933 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:10.861817 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.861847 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.862972 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.862978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.864969 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.864986 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.865781 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.865790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.866135 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.866868 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:10.866902 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:10.869948 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:10.873214 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:10.873223 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:10.873242 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:10.873865 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:10.874291 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:10.874295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:10.874297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:10.874488 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:10.875024 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:10.875032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:10.875389 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 13:15:10.875625 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 13:15:10.876179 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":329,"threads_started":1}
I20260808 13:15:10.878903 20701 heartbeater.cc:347] Connected to a master server at 127.31.59.190:44777
I20260808 13:15:10.879259 20701 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:10.880196 20701 heartbeater.cc:511] Master 127.31.59.190:44777 requested a full tablet report, sending...
I20260808 13:15:10.881786 20211 ts_manager.cc:194] Registered new tserver with Master: ee7d33aabe8f4525b36b5d62fbb2e3ae (127.31.59.131:42057)
I20260808 13:15:10.882592 20211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.131@KRBTEST.COM'} at 127.31.59.131:49889
I20260808 13:15:10.886382 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:15:10.902742 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.891001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.31.59.190:44777)
0808 13:15:10.891167 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.891171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.891249 (+ 78us) server_negotiation.cc:407] Connection header received
0808 13:15:10.891409 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.891412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.891473 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.891550 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:10.893006 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.893792 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.895006 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.895387 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.895475 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.897793 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:10.897798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:10.897801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:10.897832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:10.900313 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.900977 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.900982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.900984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.901128 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:10.901437 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:10.901440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:10.901458 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:10.901807 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:10.901988 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.902273 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.902445 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:15:10.906742 20211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:15:10.914000 20211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:15:10.937916 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.926161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:42057 (local address 127.0.0.1:45378)
0808 13:15:10.927579 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:10.927648 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:10.927826 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:10.929633 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:10.929640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:10.929670 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:10.930128 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.930142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.932670 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.932675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.934864 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.934879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.936660 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.936670 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.936958 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.937014 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 13:15:10.937145 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 13:15:10.937329 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":57,"thread_start_us":1036,"threads_started":1}
I20260808 13:15:10.938522 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.926001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.31.59.130:34557)
0808 13:15:10.927101 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.927111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.927800 (+ 689us) server_negotiation.cc:407] Connection header received
0808 13:15:10.928902 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.928910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.929123 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.929311 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:10.930894 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.932190 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.934963 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.936433 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.936978 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.937192 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.938028 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.938212 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":808,"threads_started":1}
I20260808 13:15:10.937937 20717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.925961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:34557 (local address 127.0.0.1:44262)
0808 13:15:10.927579 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:10.927649 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:10.928586 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:10.929777 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:10.929781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:10.929809 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:10.930702 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.930717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.932350 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.932355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.934707 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.934719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.936690 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.936697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.937264 (+ 567us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.937297 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:15:10.937372 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 13:15:10.937514 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"mutex_wait_us":878,"thread_start_us":1302,"threads_started":1}
I20260808 13:15:10.938913 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.928813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.31.59.129:40501)
0808 13:15:10.931331 (+ 2518us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.931340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.931365 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:15:10.931450 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.931457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.931649 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.931834 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:10.932419 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.934004 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.935927 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.937122 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.937274 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.937518 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.938429 (+ 911us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.938600 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":2170,"threads_started":1}
I20260808 13:15:10.938968 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.926847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.31.59.131:42057)
0808 13:15:10.928263 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:10.928269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:10.928293 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:15:10.928377 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:10.928383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:10.928622 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:10.930560 (+ 1938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:10.930842 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.932374 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.935041 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.936429 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.938170 (+ 1741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.938457 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:10.938532 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 13:15:10.938684 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":1127,"threads_started":1}
I20260808 13:15:10.938901 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:10.928418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51214)
0808 13:15:10.930243 (+ 1825us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:10.930259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:10.930369 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:10.931848 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:10.931853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:10.931880 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:10.932221 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.932233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.934147 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.934153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.935778 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:10.935790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:10.937615 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:10.937620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:10.937752 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:10.937783 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:15:10.937880 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:15:10.937980 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1696,"thread_start_us":1558,"threads_started":1}
I20260808 13:15:10.942399 20490 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf (DEFAULT_TABLE table=test-table [id=5f5de8faf5fb4715a302278d8c3d48c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:10.942421 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf (DEFAULT_TABLE table=test-table [id=5f5de8faf5fb4715a302278d8c3d48c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:10.942422 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf (DEFAULT_TABLE table=test-table [id=5f5de8faf5fb4715a302278d8c3d48c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:10.943590 20490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2a0011d41e4af48e1e40e93da5eebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:10.943813 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2a0011d41e4af48e1e40e93da5eebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:10.943813 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2a0011d41e4af48e1e40e93da5eebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:10.955893 20721 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Bootstrap starting.
I20260808 13:15:10.959657 20721 tablet_bootstrap.cc:654] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:10.960675 20722 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Bootstrap starting.
I20260808 13:15:10.960970 20721 log.cc:824] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:10.961676 20723 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Bootstrap starting.
I20260808 13:15:10.964792 20723 tablet_bootstrap.cc:654] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:10.964939 20722 tablet_bootstrap.cc:654] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:10.965819 20723 log.cc:824] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:10.966207 20722 log.cc:824] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:10.968816 20723 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: No bootstrap required, opened a new log
I20260808 13:15:10.969055 20721 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: No bootstrap required, opened a new log
I20260808 13:15:10.969074 20723 ts_tablet_manager.cc:1397] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:15:10.969297 20722 tablet_bootstrap.cc:492] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: No bootstrap required, opened a new log
I20260808 13:15:10.969341 20721 ts_tablet_manager.cc:1397] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260808 13:15:10.969569 20722 ts_tablet_manager.cc:1397] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 13:15:10.976662 20723 raft_consensus.cc:348] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.977054 20723 raft_consensus.cc:374] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:10.977151 20723 raft_consensus.cc:729] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89778dd6d9c94595b46cbdddbad5433b, State: Initialized, Role: FOLLOWER
I20260808 13:15:10.977782 20723 consensus_queue.cc:260] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.979583 20723 ts_tablet_manager.cc:1428] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 13:15:10.979929 20722 raft_consensus.cc:348] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.979929 20721 raft_consensus.cc:348] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.980378 20721 raft_consensus.cc:374] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:10.980373 20722 raft_consensus.cc:374] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:10.980485 20722 raft_consensus.cc:729] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7d33aabe8f4525b36b5d62fbb2e3ae, State: Initialized, Role: FOLLOWER
I20260808 13:15:10.980480 20721 raft_consensus.cc:729] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb9f385053dd42a581975a1c76082d18, State: Initialized, Role: FOLLOWER
I20260808 13:15:10.981078 20722 consensus_queue.cc:260] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.981078 20721 consensus_queue.cc:260] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:10.982247 20701 heartbeater.cc:503] Master 127.31.59.190:44777 was elected leader, sending a full tablet report...
I20260808 13:15:10.982923 20722 ts_tablet_manager.cc:1428] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260808 13:15:10.984822 20721 ts_tablet_manager.cc:1428] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260808 13:15:10.985070 20560 heartbeater.cc:503] Master 127.31.59.190:44777 was elected leader, sending a full tablet report...
W20260808 13:15:11.031574 20419 tablet.cc:2369] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:15:11.072649 20561 tablet.cc:2369] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:11.097690 20729 raft_consensus.cc:482] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:15:11.098001 20729 raft_consensus.cc:504] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:11.099777 20729 leader_election.cc:288] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7d33aabe8f4525b36b5d62fbb2e3ae (127.31.59.131:42057), 89778dd6d9c94595b46cbdddbad5433b (127.31.59.129:40501)
W20260808 13:15:11.103886 20702 tablet.cc:2369] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:11.106555 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.100734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:43647 (local address 127.31.59.129:40501)
0808 13:15:11.100952 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.100958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.101146 (+ 188us) server_negotiation.cc:407] Connection header received
0808 13:15:11.101302 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.101306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.101363 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.101431 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:11.102063 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.102785 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.104839 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.105672 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.105814 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.105890 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.106184 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.106314 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:15:11.106709 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.100216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.31.59.130:43647)
0808 13:15:11.101019 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.101052 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:11.101170 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:11.101491 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:11.101495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:11.101518 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:11.101905 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:11.101918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.102909 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.102914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:11.104689 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:11.104702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.105883 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.105889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:11.106015 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.106059 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 13:15:11.106147 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:15:11.106281 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":25,"thread_start_us":573,"threads_started":1}
I20260808 13:15:11.106755 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.100103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:54235 (local address 127.31.59.131:42057)
0808 13:15:11.100334 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.100342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.101021 (+ 679us) server_negotiation.cc:407] Connection header received
0808 13:15:11.101310 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.101316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.101378 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.101479 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:11.102144 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.102862 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.105057 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.105869 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.106021 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.106106 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.106389 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.106511 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:15:11.106886 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.100087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:42057 (local address 127.31.59.130:54235)
0808 13:15:11.100907 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.100927 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:11.101140 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:11.101506 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:11.101510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:11.101532 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:11.102006 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:11.102019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.103011 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.103015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:11.104913 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:11.104925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.106084 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.106090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:11.106247 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.106271 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:15:11.106349 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:15:11.106472 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":45,"thread_start_us":416,"threads_started":1}
I20260808 13:15:11.107821 20652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2a0011d41e4af48e1e40e93da5eebf" candidate_uuid: "cb9f385053dd42a581975a1c76082d18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" is_pre_election: true
I20260808 13:15:11.107877 20366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2a0011d41e4af48e1e40e93da5eebf" candidate_uuid: "cb9f385053dd42a581975a1c76082d18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89778dd6d9c94595b46cbdddbad5433b" is_pre_election: true
I20260808 13:15:11.108296 20652 raft_consensus.cc:2454] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb9f385053dd42a581975a1c76082d18 in term 0.
I20260808 13:15:11.108353 20366 raft_consensus.cc:2454] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb9f385053dd42a581975a1c76082d18 in term 0.
I20260808 13:15:11.108948 20443 leader_election.cc:302] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb9f385053dd42a581975a1c76082d18, ee7d33aabe8f4525b36b5d62fbb2e3ae; no voters:
I20260808 13:15:11.109333 20729 raft_consensus.cc:2781] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:15:11.109491 20729 raft_consensus.cc:482] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:15:11.109606 20729 raft_consensus.cc:3037] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:11.111809 20729 raft_consensus.cc:504] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:11.112412 20729 leader_election.cc:288] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [CANDIDATE]: Term 1 election: Requested vote from peers ee7d33aabe8f4525b36b5d62fbb2e3ae (127.31.59.131:42057), 89778dd6d9c94595b46cbdddbad5433b (127.31.59.129:40501)
I20260808 13:15:11.112860 20652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2a0011d41e4af48e1e40e93da5eebf" candidate_uuid: "cb9f385053dd42a581975a1c76082d18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae"
I20260808 13:15:11.112890 20366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2a0011d41e4af48e1e40e93da5eebf" candidate_uuid: "cb9f385053dd42a581975a1c76082d18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89778dd6d9c94595b46cbdddbad5433b"
I20260808 13:15:11.113070 20652 raft_consensus.cc:3037] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:11.113070 20366 raft_consensus.cc:3037] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:11.116225 20366 raft_consensus.cc:2454] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb9f385053dd42a581975a1c76082d18 in term 1.
I20260808 13:15:11.116225 20652 raft_consensus.cc:2454] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb9f385053dd42a581975a1c76082d18 in term 1.
I20260808 13:15:11.116631 20443 leader_election.cc:302] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb9f385053dd42a581975a1c76082d18, ee7d33aabe8f4525b36b5d62fbb2e3ae; no voters:
I20260808 13:15:11.116888 20729 raft_consensus.cc:2781] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:11.117373 20729 raft_consensus.cc:686] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 1 LEADER]: Becoming Leader. State: Replica: cb9f385053dd42a581975a1c76082d18, State: Running, Role: LEADER
I20260808 13:15:11.117916 20729 consensus_queue.cc:237] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } }
I20260808 13:15:11.122072 20211 catalog_manager.cc:5495] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb9f385053dd42a581975a1c76082d18 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "cb9f385053dd42a581975a1c76082d18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb9f385053dd42a581975a1c76082d18" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 34557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 } health_report { overall_health: UNKNOWN } } }
I20260808 13:15:11.172268 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.166803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.31.59.130:34557)
0808 13:15:11.167015 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.167021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.167109 (+ 88us) server_negotiation.cc:407] Connection header received
0808 13:15:11.167344 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.167348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.167440 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.167572 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:11.167980 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.168714 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.170139 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.170588 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.170694 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.171203 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:11.171708 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:11.171844 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.171928 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.172012 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:15:11.180616 20366 raft_consensus.cc:1261] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 1 FOLLOWER]: Refusing update from remote peer cb9f385053dd42a581975a1c76082d18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:11.180626 20652 raft_consensus.cc:1261] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 1 FOLLOWER]: Refusing update from remote peer cb9f385053dd42a581975a1c76082d18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:11.181772 20733 consensus_queue.cc:1035] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89778dd6d9c94595b46cbdddbad5433b" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:11.182549 20733 consensus_queue.cc:1035] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7d33aabe8f4525b36b5d62fbb2e3ae" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:11.194542 20737 mvcc.cc:205] Tried to move back new op lower bound from 7316254356184805376 to 7316254355942514688. Current Snapshot: MvccSnapshot[applied={T|T < 7316254356184805376}]
I20260808 13:15:11.194871 20736 mvcc.cc:205] Tried to move back new op lower bound from 7316254356184805376 to 7316254355942514688. Current Snapshot: MvccSnapshot[applied={T|T < 7316254356184805376}]
I20260808 13:15:11.209298 20210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:11.209649 20210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:11.213687 20210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 277a8e0bbc24418a8a4959d61b9c9587: Sending DeleteTablet for 3 replicas of tablet 3b2a0011d41e4af48e1e40e93da5eebf
I20260808 13:15:11.214866 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:11 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:45378
I20260808 13:15:11.215196 20490 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:11 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:44262
I20260808 13:15:11.215909 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2a0011d41e4af48e1e40e93da5eebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:11 UTC) from {username='kudu', principal='kudu/127.31.59.190@KRBTEST.COM'} at 127.0.0.1:51214
I20260808 13:15:11.216194 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20276
I20260808 13:15:11.228003 20748 tablet_replica.cc:330] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: stopping tablet replica
I20260808 13:15:11.228701 20747 tablet_replica.cc:330] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: stopping tablet replica
I20260808 13:15:11.229420 20748 raft_consensus.cc:2229] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:11.230075 20747 raft_consensus.cc:2229] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:15:11.230901 20747 raft_consensus.cc:2258] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:11.231545 20748 raft_consensus.cc:2258] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:11.236090 20748 ts_tablet_manager.cc:1905] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:11.237310 20747 ts_tablet_manager.cc:1905] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 13:15:11 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.129@KRBTEST.COM
I20260808 13:15:11.242041 20746 tablet_replica.cc:330] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: stopping tablet replica
I20260808 13:15:11.242431 20746 raft_consensus.cc:2229] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:11.242887 20746 raft_consensus.cc:2258] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:11.245080 20746 ts_tablet_manager.cc:1905] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:11.247184 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.217169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.31.59.129:40501)
0808 13:15:11.217340 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.217346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.227006 (+ 9660us) server_negotiation.cc:407] Connection header received
0808 13:15:11.227229 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.227236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.227331 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.227444 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:11.229596 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.230751 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.232672 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.233368 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.233495 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.239316 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:11.239333 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:11.239353 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:11.239399 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:11.243310 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.244272 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.244282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.244286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.244469 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.244946 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.244951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.244955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.245436 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:11.245798 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.246478 (+ 680us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.246751 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:15:11.248687 20413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:11.255092 20747 ts_tablet_manager.cc:1918] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:11.255311 20747 log.cc:1196] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/3b2a0011d41e4af48e1e40e93da5eebf
I20260808 13:15:11.255460 20746 ts_tablet_manager.cc:1918] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:11.255623 20746 log.cc:1196] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/3b2a0011d41e4af48e1e40e93da5eebf
I20260808 13:15:11.255901 20747 ts_tablet_manager.cc:1939] T 3b2a0011d41e4af48e1e40e93da5eebf P cb9f385053dd42a581975a1c76082d18: Deleting consensus metadata
I20260808 13:15:11.256143 20746 ts_tablet_manager.cc:1939] T 3b2a0011d41e4af48e1e40e93da5eebf P ee7d33aabe8f4525b36b5d62fbb2e3ae: Deleting consensus metadata
I20260808 13:15:11.257942 20194 catalog_manager.cc:4841] TS cb9f385053dd42a581975a1c76082d18 (127.31.59.130:34557): tablet 3b2a0011d41e4af48e1e40e93da5eebf (table test-table [id=5f5de8faf5fb4715a302278d8c3d48c9]) successfully deleted
I20260808 13:15:11.258286 20197 catalog_manager.cc:4841] TS ee7d33aabe8f4525b36b5d62fbb2e3ae (127.31.59.131:42057): tablet 3b2a0011d41e4af48e1e40e93da5eebf (table test-table [id=5f5de8faf5fb4715a302278d8c3d48c9]) successfully deleted
I20260808 13:15:11.400988 20748 ts_tablet_manager.cc:1918] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:11.401252 20748 log.cc:1196] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/3b2a0011d41e4af48e1e40e93da5eebf
I20260808 13:15:11.401855 20748 ts_tablet_manager.cc:1939] T 3b2a0011d41e4af48e1e40e93da5eebf P 89778dd6d9c94595b46cbdddbad5433b: Deleting consensus metadata
I20260808 13:15:11.401827 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20276
W20260808 13:15:11.428021 20194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.129:40501 (error 108)
I20260808 13:15:11.428161 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20424
I20260808 13:15:11.430073 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.429586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51244)
0808 13:15:11.429748 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.429889 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 13:15:11.430349 20194 catalog_manager.cc:4584] TS 89778dd6d9c94595b46cbdddbad5433b (127.31.59.129:40501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b2a0011d41e4af48e1e40e93da5eebf: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Aug 08 13:15:11 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.130@KRBTEST.COM
I20260808 13:15:11.445622 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.429550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44272 (local address 127.31.59.130:34557)
0808 13:15:11.429709 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.429714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.429732 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:15:11.429799 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.429804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.429907 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.430016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:11.431723 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.432535 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.434379 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.434799 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.434892 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.439874 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:11.439890 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:11.439904 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:11.439949 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:11.442951 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.443579 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.443586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.443589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.443736 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.444093 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.444097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.444099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.444461 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:11.444787 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.445028 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.445247 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 13:15:11.446261 20555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:11.484071 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.483524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51260)
0808 13:15:11.483768 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.483894 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 13:15:11.485153 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.484777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51266)
0808 13:15:11.484916 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.485002 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 13:15:11.545763 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.545124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51270)
0808 13:15:11.545382 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.545535 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 13:15:11.546808 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.546437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51286)
0808 13:15:11.546583 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.546671 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:15:11.563086 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20424
I20260808 13:15:11.584810 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20566
Aug 08 13:15:11 dist-test-slave-49h3 krb5kdc[20162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.131@KRBTEST.COM
I20260808 13:15:11.599483 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.585597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.31.59.131:42057)
0808 13:15:11.585779 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:11.585786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:11.585859 (+ 73us) server_negotiation.cc:407] Connection header received
0808 13:15:11.586061 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:11.586065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:11.586143 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:11.586243 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:11.587552 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.588275 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.589580 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:11.589974 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:11.590067 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:11.594587 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:11.594604 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:11.594621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:11.594655 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:11.597117 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.597822 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.597828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.597831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.597968 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:11.598269 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:11.598271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:11.598273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:11.598608 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:11.598839 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:11.599055 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 13:15:11.599220 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:15:11.600092 20695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:11.627662 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.627148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51288)
0808 13:15:11.627371 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.627471 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 13:15:11.628731 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.628356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51292)
0808 13:15:11.628544 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.628622 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 13:15:11.689383 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20566
I20260808 13:15:11.707538 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20176
I20260808 13:15:11.708727 20244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:11.819983 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:11.819337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:40501 (local address 127.0.0.1:51294)
0808 13:15:11.819695 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:11.819811 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.129:40501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 13:15:11.820101 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20176
2026-08-08T13:15:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4472 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T13:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:11Z Disabled control of system clock
I20260808 13:15:11.912010 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37233
--webserver_interface=127.31.59.190
--webserver_port=0
--builtin_ntp_servers=127.31.59.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:37233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:15:12.251251 20759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:12.251545 20759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:12.251677 20759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:12.258999 20759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:12.259135 20759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:12.259203 20759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:12.259276 20759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:12.259338 20759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:12.268752 20759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:37233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:12.270165 20759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:12.272002 20759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:12.283382 20764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:12.283391 20767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:12.283422 20765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:12.284089 20759 server_base.cc:1034] running on GCE node
I20260808 13:15:12.284826 20759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:12.286190 20759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:12.287420 20759 hybrid_clock.cc:648] HybridClock initialized: now 1786194912287367 us; error 38 us; skew 500 ppm
I20260808 13:15:12.287778 20759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:12.290128 20759 webserver.cc:460] Webserver started at http://127.31.59.190:39667/ using document root <none> and password file <none>
I20260808 13:15:12.290797 20759 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:12.290897 20759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:12.291211 20759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:12.294150 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "d753bf003133421cb4e2a7e58129f363"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "ca50a60850b73ba70f7b120dea45b4e4"
server_key_iv: "ec457eb6127c8c81d53f8754fbc3fbc3"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.295012 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "d753bf003133421cb4e2a7e58129f363"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "ca50a60850b73ba70f7b120dea45b4e4"
server_key_iv: "ec457eb6127c8c81d53f8754fbc3fbc3"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.300017 20759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:12.303431 20774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:12.304702 20759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:12.304869 20759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "d753bf003133421cb4e2a7e58129f363"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "ca50a60850b73ba70f7b120dea45b4e4"
server_key_iv: "ec457eb6127c8c81d53f8754fbc3fbc3"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.305073 20759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:12.324723 20759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:12.325587 20759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:12.325857 20759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:12.343381 20759 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:37233
I20260808 13:15:12.343405 20832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:37233 every 8 connection(s)
I20260808 13:15:12.344678 20759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:12.348243 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:12.352033 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20759
I20260808 13:15:12.352200 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:12.352629 31982 external_mini_cluster.cc:1428] Setting key e07a8c227a9d118d25513827c06f9ece
I20260808 13:15:12.360167 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: Bootstrap starting.
I20260808 13:15:12.363756 20833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:12.365016 20833 log.cc:824] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:12.368161 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:12.354386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.31.59.190:37233)
0808 13:15:12.355444 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:12.355459 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:12.355512 (+ 53us) server_negotiation.cc:407] Connection header received
0808 13:15:12.358518 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:12.358547 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:12.359108 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:12.359514 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:12.360057 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.362033 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.363298 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.364330 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.364502 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:12.365624 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:12.365634 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:12.365654 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:12.366231 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:12.366320 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:12.366354 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:12.366466 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:12.366648 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 13:15:12.366961 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":356,"threads_started":1}
I20260808 13:15:12.368296 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: No bootstrap required, opened a new log
I20260808 13:15:12.376041 20833 raft_consensus.cc:348] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } }
I20260808 13:15:12.376403 20833 raft_consensus.cc:374] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:12.376506 20833 raft_consensus.cc:729] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d753bf003133421cb4e2a7e58129f363, State: Initialized, Role: FOLLOWER
I20260808 13:15:12.377101 20833 consensus_queue.cc:260] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } }
I20260808 13:15:12.377372 20833 raft_consensus.cc:388] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:12.377522 20833 raft_consensus.cc:482] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:12.377682 20833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:12.379848 20833 raft_consensus.cc:504] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } }
I20260808 13:15:12.380336 20833 leader_election.cc:302] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d753bf003133421cb4e2a7e58129f363; no voters:
I20260808 13:15:12.380893 20833 leader_election.cc:288] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:12.381044 20839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:12.381721 20839 raft_consensus.cc:686] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [term 1 LEADER]: Becoming Leader. State: Replica: d753bf003133421cb4e2a7e58129f363, State: Running, Role: LEADER
I20260808 13:15:12.382210 20839 consensus_queue.cc:237] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } }
I20260808 13:15:12.382929 20833 sys_catalog.cc:564] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:12.384696 20841 sys_catalog.cc:455] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d753bf003133421cb4e2a7e58129f363. Latest consensus state: current_term: 1 leader_uuid: "d753bf003133421cb4e2a7e58129f363" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } } }
I20260808 13:15:12.385025 20841 sys_catalog.cc:458] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:12.385613 20840 sys_catalog.cc:455] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d753bf003133421cb4e2a7e58129f363" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753bf003133421cb4e2a7e58129f363" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 37233 } } }
I20260808 13:15:12.385984 20840 sys_catalog.cc:458] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:12.387305 20845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:12.392974 20845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:12.402361 20845 catalog_manager.cc:1347] Generated new cluster ID: 257967de4c364d23a6817425e39da8f1
I20260808 13:15:12.402480 20845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:12.413333 20845 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:12.414525 20845 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:12.423252 20845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: Generated new TSK 0
I20260808 13:15:12.424258 20845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:12.435966 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.129:0
--local_ip_for_outbound_sockets=127.31.59.129
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--builtin_ntp_servers=127.31.59.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:15:12.776080 20861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:12.776441 20861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:12.776583 20861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:12.783994 20861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:12.784103 20861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:12.784258 20861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.129
I20260808 13:15:12.793644 20861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.129
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.129
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:12.794914 20861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:12.796733 20861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:12.809540 20869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:12.809720 20868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:12.809674 20871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:12.810734 20861 server_base.cc:1034] running on GCE node
I20260808 13:15:12.811357 20861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:12.812314 20861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:12.813529 20861 hybrid_clock.cc:648] HybridClock initialized: now 1786194912813481 us; error 33 us; skew 500 ppm
I20260808 13:15:12.813853 20861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:12.816097 20861 webserver.cc:460] Webserver started at http://127.31.59.129:42731/ using document root <none> and password file <none>
I20260808 13:15:12.816771 20861 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:12.816875 20861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:12.817199 20861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:12.820153 20861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data/instance:
uuid: "e84edc32697e425d9d7e56249a5d390f"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "f6eb1262a06fa2cb7b4214712f522a73"
server_key_iv: "db785b06d6412aab7c5793e7e0129bb5"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.820940 20861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance:
uuid: "e84edc32697e425d9d7e56249a5d390f"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "f6eb1262a06fa2cb7b4214712f522a73"
server_key_iv: "db785b06d6412aab7c5793e7e0129bb5"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.825934 20861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 13:15:12.829360 20877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:12.830672 20861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:12.830835 20861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal
uuid: "e84edc32697e425d9d7e56249a5d390f"
format_stamp: "Formatted at 2026-08-08 13:15:12 on dist-test-slave-49h3"
server_key: "f6eb1262a06fa2cb7b4214712f522a73"
server_key_iv: "db785b06d6412aab7c5793e7e0129bb5"
server_key_version: "encryptionkey@0"
I20260808 13:15:12.831034 20861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:12.866403 20861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:12.867449 20861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:12.867828 20861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:12.869072 20861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:12.870576 20861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:12.870674 20861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:12.870770 20861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:12.870824 20861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:12.915827 20861 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.129:36231
I20260808 13:15:12.916000 20994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.129:36231 every 8 connection(s)
I20260808 13:15:12.917425 20861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/data/info.pb
I20260808 13:15:12.918988 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20861
I20260808 13:15:12.919199 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal/instance
I20260808 13:15:12.919788 31982 external_mini_cluster.cc:1428] Setting key dcc138488a4588e151683e5b05780059
I20260808 13:15:12.922801 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.130:0
--local_ip_for_outbound_sockets=127.31.59.130
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--builtin_ntp_servers=127.31.59.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:15:12.934370 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:12.922144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.129:49171 (local address 127.31.59.190:37233)
0808 13:15:12.922812 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:12.922818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:12.926539 (+ 3721us) server_negotiation.cc:407] Connection header received
0808 13:15:12.926606 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:12.926610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:12.926686 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:12.926780 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:12.928245 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.929155 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.931549 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.932079 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.932212 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:12.933491 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:12.933497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:12.933502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:12.933525 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:12.933573 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:12.933588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:12.933673 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:12.933866 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 13:15:12.933966 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":323,"threads_started":1}
I20260808 13:15:12.935338 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:12.922893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37233 (local address 127.31.59.129:49171)
0808 13:15:12.923979 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:12.924035 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:12.926196 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:12.927072 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:12.927084 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:12.927206 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:12.928044 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:12.928069 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.929340 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.929346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:12.931394 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:12.931408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:12.932286 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:12.932290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:12.932490 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:12.933153 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:12.933156 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:12.933196 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:12.933282 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:12.933671 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:12.933679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:12.933685 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:15:12.933851 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:15:12.934162 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":485,"threads_started":1}
I20260808 13:15:12.936956 20995 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37233
I20260808 13:15:12.937258 20995 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:12.938030 20995 heartbeater.cc:511] Master 127.31.59.190:37233 requested a full tablet report, sending...
I20260808 13:15:12.940301 20793 ts_manager.cc:194] Registered new tserver with Master: e84edc32697e425d9d7e56249a5d390f (127.31.59.129:36231)
I20260808 13:15:12.942262 20793 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.129:49171
W20260808 13:15:13.267566 20999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:13.267846 20999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:13.267935 20999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:13.275239 20999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:13.275347 20999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:13.275455 20999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.130
I20260808 13:15:13.285174 20999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.130
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.130
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:13.286520 20999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:13.288296 20999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:13.300756 21009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:13.300832 21004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:13.300796 21005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:13.301524 20999 server_base.cc:1034] running on GCE node
I20260808 13:15:13.302048 20999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:13.302938 20999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:13.304176 20999 hybrid_clock.cc:648] HybridClock initialized: now 1786194913304112 us; error 54 us; skew 500 ppm
I20260808 13:15:13.304469 20999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:13.306810 20999 webserver.cc:460] Webserver started at http://127.31.59.130:33637/ using document root <none> and password file <none>
I20260808 13:15:13.307488 20999 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:13.307583 20999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:13.307891 20999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:13.310900 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data/instance:
uuid: "0f5e80a1f281433d846956c9cdcac7a2"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "d2893421b4e66a6cd19e44bb24bc1edb"
server_key_iv: "92ca24df3b1b705846806006e59a8ccd"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.311715 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance:
uuid: "0f5e80a1f281433d846956c9cdcac7a2"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "d2893421b4e66a6cd19e44bb24bc1edb"
server_key_iv: "92ca24df3b1b705846806006e59a8ccd"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.316708 20999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 13:15:13.320068 21015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.321311 20999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:13.321501 20999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal
uuid: "0f5e80a1f281433d846956c9cdcac7a2"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "d2893421b4e66a6cd19e44bb24bc1edb"
server_key_iv: "92ca24df3b1b705846806006e59a8ccd"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.321707 20999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:13.347849 20999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:13.348675 20999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:13.348958 20999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:13.349973 20999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:13.351482 20999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:13.351580 20999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.351699 20999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:13.351775 20999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.382833 20999 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.130:42067
I20260808 13:15:13.382890 21138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.130:42067 every 8 connection(s)
I20260808 13:15:13.384254 20999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/data/info.pb
I20260808 13:15:13.388520 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 20999
I20260808 13:15:13.388729 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal/instance
I20260808 13:15:13.389307 31982 external_mini_cluster.cc:1428] Setting key f8a31e0b9ecc4046fbb46e910e9634f1
I20260808 13:15:13.392398 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.131:0
--local_ip_for_outbound_sockets=127.31.59.131
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--builtin_ntp_servers=127.31.59.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:15:13.397686 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.386799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:47505 (local address 127.31.59.190:37233)
0808 13:15:13.386973 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.386978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.388457 (+ 1479us) server_negotiation.cc:407] Connection header received
0808 13:15:13.390762 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.390769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.390842 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.390941 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:13.392181 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.392953 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.395035 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.395454 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.395544 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.396987 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:13.396992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:13.396995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:13.397012 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:13.397044 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:13.397055 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:13.397143 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.397344 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.397422 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:15:13.398932 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.387302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37233 (local address 127.31.59.130:47505)
0808 13:15:13.388276 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:13.388332 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:13.390508 (+ 2176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:13.391139 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:13.391150 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:13.391262 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:13.391992 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.392015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.393107 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.393112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.394882 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.394897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.395672 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.395678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.395952 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.396646 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:13.396649 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:13.396698 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:13.396796 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:13.397139 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:13.397149 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:13.397157 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:15:13.397323 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:15:13.397720 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":297,"threads_started":1}
I20260808 13:15:13.400259 21139 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37233
I20260808 13:15:13.400550 21139 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:13.401284 21139 heartbeater.cc:511] Master 127.31.59.190:37233 requested a full tablet report, sending...
I20260808 13:15:13.402657 20793 ts_manager.cc:194] Registered new tserver with Master: 0f5e80a1f281433d846956c9cdcac7a2 (127.31.59.130:42067)
I20260808 13:15:13.403461 20793 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.130:47505
W20260808 13:15:13.726071 21142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:13.726361 21142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:13.726490 21142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:13.733839 21142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:13.733981 21142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:13.734153 21142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.131
I20260808 13:15:13.743675 21142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.131
--webserver_port=0
--tserver_master_addrs=127.31.59.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.131
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:13.745004 21142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:13.746917 21142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:13.759498 21147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:13.759562 21150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:13.759590 21148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:13.760470 21142 server_base.cc:1034] running on GCE node
I20260808 13:15:13.761065 21142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:13.762079 21142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:13.763286 21142 hybrid_clock.cc:648] HybridClock initialized: now 1786194913763231 us; error 43 us; skew 500 ppm
I20260808 13:15:13.763585 21142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:15:13.765777 21142 webserver.cc:460] Webserver started at http://127.31.59.131:33211/ using document root <none> and password file <none>
I20260808 13:15:13.766477 21142 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:13.766582 21142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:13.766891 21142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:13.769778 21142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data/instance:
uuid: "59bebce78daf470b979de3ae363dc95b"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "f0a1122a7ed5666265e13acd4d122899"
server_key_iv: "d1636267c90317ce1965a104e82a80f4"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.770562 21142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance:
uuid: "59bebce78daf470b979de3ae363dc95b"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "f0a1122a7ed5666265e13acd4d122899"
server_key_iv: "d1636267c90317ce1965a104e82a80f4"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.775619 21142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:15:13.779124 21157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.780386 21142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:15:13.780539 21142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal
uuid: "59bebce78daf470b979de3ae363dc95b"
format_stamp: "Formatted at 2026-08-08 13:15:13 on dist-test-slave-49h3"
server_key: "f0a1122a7ed5666265e13acd4d122899"
server_key_iv: "d1636267c90317ce1965a104e82a80f4"
server_key_version: "encryptionkey@0"
I20260808 13:15:13.780753 21142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:13.825537 21142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:13.826520 21142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:13.826802 21142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:13.827749 21142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:15:13.829277 21142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:15:13.829380 21142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.829505 21142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:15:13.829571 21142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:13.860117 21142 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.131:37581
I20260808 13:15:13.860168 21282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.131:37581 every 8 connection(s)
I20260808 13:15:13.861385 21142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/data/info.pb
I20260808 13:15:13.864706 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21142
I20260808 13:15:13.864925 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal/instance
I20260808 13:15:13.865486 31982 external_mini_cluster.cc:1428] Setting key da8b380054ff4c484fcb10e7673802b3
I20260808 13:15:13.873934 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.863908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.131:56521 (local address 127.31.59.190:37233)
0808 13:15:13.864078 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.864082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.865669 (+ 1587us) server_negotiation.cc:407] Connection header received
0808 13:15:13.867893 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.867899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.867986 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.868089 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:13.869212 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.870025 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.871508 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.871907 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.872026 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.873214 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:13.873219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:13.873222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:13.873240 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:13.873272 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:13.873284 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:13.873355 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.873584 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.873676 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:15:13.874903 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.864368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:37233 (local address 127.31.59.131:56521)
0808 13:15:13.865471 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:13.865534 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:13.867629 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:13.868324 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:13.868336 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:13.868459 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:13.869050 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.869067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.870150 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.870153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.871391 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.871402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.872079 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.872084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.872286 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.872914 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:15:13.872917 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:13.872957 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:15:13.873039 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:13.873359 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:13.873365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:13.873370 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:15:13.873552 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 13:15:13.873864 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":328,"threads_started":1}
I20260808 13:15:13.876230 21283 heartbeater.cc:347] Connected to a master server at 127.31.59.190:37233
I20260808 13:15:13.876493 21283 heartbeater.cc:464] Registering TS with master...
I20260808 13:15:13.877242 21283 heartbeater.cc:511] Master 127.31.59.190:37233 requested a full tablet report, sending...
I20260808 13:15:13.878651 20793 ts_manager.cc:194] Registered new tserver with Master: 59bebce78daf470b979de3ae363dc95b (127.31.59.131:37581)
I20260808 13:15:13.879642 20793 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.131:56521
I20260808 13:15:13.880604 31982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:15:13.890208 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.884653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.31.59.190:37233)
0808 13:15:13.884899 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.884906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.885341 (+ 435us) server_negotiation.cc:407] Connection header received
0808 13:15:13.885502 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.885506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.885606 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.885694 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:13.886194 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.886858 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.888121 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.888576 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.888667 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.889613 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:13.889619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:13.889622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:13.889639 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:13.889669 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:13.889688 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:13.889754 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.889837 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.889904 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:15:13.893687 20793 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:15:13.900391 20793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:15:13.942761 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.912510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.130:42067 (local address 127.0.0.1:43000)
0808 13:15:13.931538 (+ 19028us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:13.931554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:13.931678 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:13.932605 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:13.932613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:13.932642 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:13.932973 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.932982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.934866 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.934872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.939427 (+ 4555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.939444 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.941710 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.941717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.941950 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.942005 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 13:15:13.942135 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 13:15:13.942351 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18890,"mutex_wait_us":50,"thread_start_us":18748,"threads_started":1}
I20260808 13:15:13.943439 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.913498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37581 (local address 127.0.0.1:56444)
0808 13:15:13.933492 (+ 19994us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:13.933510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:13.933622 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:13.934264 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:13.934269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:13.934296 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:13.934697 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.934713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.936396 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.936402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.940328 (+ 3926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.940341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.942625 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.942632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.942822 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.942859 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:15:13.942942 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:15:13.943072 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19855,"mutex_wait_us":63,"thread_start_us":16699,"threads_started":1}
I20260808 13:15:13.943629 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.912698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.31.59.130:42067)
0808 13:15:13.929095 (+ 16397us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.929106 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.931813 (+ 2707us) server_negotiation.cc:407] Connection header received
0808 13:15:13.931922 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.931929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.932187 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.932369 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:13.933137 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.934716 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.939700 (+ 4984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.941149 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.941288 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.941569 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.943193 (+ 1624us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.943337 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16184,"thread_start_us":16400,"threads_started":1}
I20260808 13:15:13.944808 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.912367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36231 (local address 127.0.0.1:51950)
0808 13:15:13.930602 (+ 18235us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:13.930661 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:13.930870 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:13.934633 (+ 3763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:13.934637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:13.934665 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:13.936861 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.936874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.938804 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.938809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.942250 (+ 3441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:13.942265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.944120 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.944128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:13.944316 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.944350 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:15:13.944444 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 13:15:13.944567 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17984,"thread_start_us":20850,"threads_started":1}
I20260808 13:15:13.945286 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.914041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.31.59.131:37581)
0808 13:15:13.930004 (+ 15963us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.930013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.933746 (+ 3733us) server_negotiation.cc:407] Connection header received
0808 13:15:13.933856 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.933863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.934063 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.934257 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:13.935163 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.936253 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.940579 (+ 4326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.942072 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.942198 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.942494 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.944736 (+ 2242us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.944910 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15796,"thread_start_us":15644,"threads_started":1}
I20260808 13:15:13.945717 20995 heartbeater.cc:503] Master 127.31.59.190:37233 was elected leader, sending a full tablet report...
I20260808 13:15:13.946499 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:13.912499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.31.59.129:36231)
0808 13:15:13.933840 (+ 21341us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:13.933847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:13.933872 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:15:13.933947 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:13.933952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:13.934145 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:13.934385 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:13.937058 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.938637 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.942476 (+ 3839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:13.943907 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:13.944058 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:13.944316 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:13.945134 (+ 818us) server_negotiation.cc:299] Negotiation successful
0808 13:15:13.945331 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21186,"thread_start_us":21066,"threads_started":1}
I20260808 13:15:13.947868 21068 tablet_service.cc:1467] Processing CreateTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 (DEFAULT_TABLE table=test-table [id=42329ef423a04bf99ef69478241fd394]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:13.948700 21213 tablet_service.cc:1467] Processing CreateTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 (DEFAULT_TABLE table=test-table [id=42329ef423a04bf99ef69478241fd394]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:13.949361 21068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac8851c82cc4d05ac36c928a60c6ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:13.949342 20927 tablet_service.cc:1467] Processing CreateTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 (DEFAULT_TABLE table=test-table [id=42329ef423a04bf99ef69478241fd394]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:15:13.950315 21213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac8851c82cc4d05ac36c928a60c6ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:13.950505 20927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac8851c82cc4d05ac36c928a60c6ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:13.962109 21303 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Bootstrap starting.
I20260808 13:15:13.964005 21305 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Bootstrap starting.
I20260808 13:15:13.965544 21307 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Bootstrap starting.
I20260808 13:15:13.966352 21303 tablet_bootstrap.cc:654] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:13.967558 21305 tablet_bootstrap.cc:654] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:13.968760 21305 log.cc:824] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:13.968811 21303 log.cc:824] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:13.969825 21307 tablet_bootstrap.cc:654] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:13.970769 21307 log.cc:824] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:13.973309 21307 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: No bootstrap required, opened a new log
I20260808 13:15:13.973601 21307 ts_tablet_manager.cc:1397] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:15:13.976138 21303 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: No bootstrap required, opened a new log
I20260808 13:15:13.976423 21303 ts_tablet_manager.cc:1397] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260808 13:15:13.976897 21305 tablet_bootstrap.cc:492] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: No bootstrap required, opened a new log
I20260808 13:15:13.977172 21305 ts_tablet_manager.cc:1397] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260808 13:15:13.981563 21307 raft_consensus.cc:348] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.981959 21307 raft_consensus.cc:374] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:13.982052 21307 raft_consensus.cc:729] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59bebce78daf470b979de3ae363dc95b, State: Initialized, Role: FOLLOWER
I20260808 13:15:13.982658 21307 consensus_queue.cc:260] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.983865 21283 heartbeater.cc:503] Master 127.31.59.190:37233 was elected leader, sending a full tablet report...
I20260808 13:15:13.984632 21307 ts_tablet_manager.cc:1428] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 13:15:13.986804 21303 raft_consensus.cc:348] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.987232 21303 raft_consensus.cc:374] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:13.987342 21303 raft_consensus.cc:729] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84edc32697e425d9d7e56249a5d390f, State: Initialized, Role: FOLLOWER
I20260808 13:15:13.987939 21303 consensus_queue.cc:260] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.987978 21305 raft_consensus.cc:348] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.988379 21305 raft_consensus.cc:374] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:13.988487 21305 raft_consensus.cc:729] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5e80a1f281433d846956c9cdcac7a2, State: Initialized, Role: FOLLOWER
I20260808 13:15:13.989274 21305 consensus_queue.cc:260] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:13.989813 21303 ts_tablet_manager.cc:1428] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Time spent starting tablet: real 0.013s user 0.015s sys 0.001s
I20260808 13:15:13.991005 21139 heartbeater.cc:503] Master 127.31.59.190:37233 was elected leader, sending a full tablet report...
I20260808 13:15:13.991465 21305 ts_tablet_manager.cc:1428] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260808 13:15:14.046630 21314 raft_consensus.cc:482] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:15:14.046947 21314 raft_consensus.cc:504] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:14.048643 21314 leader_election.cc:288] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e84edc32697e425d9d7e56249a5d390f (127.31.59.129:36231), 59bebce78daf470b979de3ae363dc95b (127.31.59.131:37581)
I20260808 13:15:14.055192 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.049184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:59293 (local address 127.31.59.131:37581)
0808 13:15:14.049395 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.049401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.049912 (+ 511us) server_negotiation.cc:407] Connection header received
0808 13:15:14.050109 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.050112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.050173 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.050247 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:14.050816 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.051515 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.053509 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.054276 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.054422 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.054490 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.054836 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.054960 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:15:14.055192 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.049093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.130:41623 (local address 127.31.59.129:36231)
0808 13:15:14.049313 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.049318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.050102 (+ 784us) server_negotiation.cc:407] Connection header received
0808 13:15:14.050267 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.050271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.050340 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.050416 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:15:14.050994 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.051668 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.053433 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.054238 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.054390 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.054460 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.054817 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.054945 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:15:14.055351 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.049156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.129:36231 (local address 127.31.59.130:41623)
0808 13:15:14.049991 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:14.050006 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:14.050130 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:14.050458 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:14.050462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:14.050489 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:14.050844 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:14.050853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.051818 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.051821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:14.053246 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:14.053258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.054439 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.054445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:14.054632 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.054683 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 13:15:14.054772 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 13:15:14.054926 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":69,"thread_start_us":543,"threads_started":1}
I20260808 13:15:14.055380 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.049056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.131:37581 (local address 127.31.59.130:59293)
0808 13:15:14.049786 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:14.049807 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:14.049966 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:14.050276 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:14.050280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:14.050305 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:15:14.050667 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:14.050678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.051639 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.051645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:14.053240 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:14.053259 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.054455 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.054461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:14.054660 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.054699 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:15:14.054775 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 13:15:14.054912 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":49,"thread_start_us":404,"threads_started":1}
I20260808 13:15:14.056332 20948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac8851c82cc4d05ac36c928a60c6ab8" candidate_uuid: "0f5e80a1f281433d846956c9cdcac7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84edc32697e425d9d7e56249a5d390f" is_pre_election: true
I20260808 13:15:14.056373 21233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac8851c82cc4d05ac36c928a60c6ab8" candidate_uuid: "0f5e80a1f281433d846956c9cdcac7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59bebce78daf470b979de3ae363dc95b" is_pre_election: true
I20260808 13:15:14.056834 21233 raft_consensus.cc:2454] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f5e80a1f281433d846956c9cdcac7a2 in term 0.
I20260808 13:15:14.056833 20948 raft_consensus.cc:2454] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f5e80a1f281433d846956c9cdcac7a2 in term 0.
I20260808 13:15:14.057420 21018 leader_election.cc:302] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5e80a1f281433d846956c9cdcac7a2, 59bebce78daf470b979de3ae363dc95b; no voters:
I20260808 13:15:14.057821 21314 raft_consensus.cc:2781] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:15:14.057950 21314 raft_consensus.cc:482] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:15:14.058044 21314 raft_consensus.cc:3037] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:14.060437 21314 raft_consensus.cc:504] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:14.060977 21314 leader_election.cc:288] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [CANDIDATE]: Term 1 election: Requested vote from peers e84edc32697e425d9d7e56249a5d390f (127.31.59.129:36231), 59bebce78daf470b979de3ae363dc95b (127.31.59.131:37581)
I20260808 13:15:14.061496 20948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac8851c82cc4d05ac36c928a60c6ab8" candidate_uuid: "0f5e80a1f281433d846956c9cdcac7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84edc32697e425d9d7e56249a5d390f"
I20260808 13:15:14.061496 21233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac8851c82cc4d05ac36c928a60c6ab8" candidate_uuid: "0f5e80a1f281433d846956c9cdcac7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59bebce78daf470b979de3ae363dc95b"
I20260808 13:15:14.061710 21233 raft_consensus.cc:3037] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:14.061710 20948 raft_consensus.cc:3037] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:14.064899 20948 raft_consensus.cc:2454] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f5e80a1f281433d846956c9cdcac7a2 in term 1.
I20260808 13:15:14.064899 21233 raft_consensus.cc:2454] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f5e80a1f281433d846956c9cdcac7a2 in term 1.
I20260808 13:15:14.065328 21018 leader_election.cc:302] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f5e80a1f281433d846956c9cdcac7a2, 59bebce78daf470b979de3ae363dc95b, e84edc32697e425d9d7e56249a5d390f; no voters:
I20260808 13:15:14.065671 21314 raft_consensus.cc:2781] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:14.066128 21314 raft_consensus.cc:686] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 1 LEADER]: Becoming Leader. State: Replica: 0f5e80a1f281433d846956c9cdcac7a2, State: Running, Role: LEADER
I20260808 13:15:14.066597 21314 consensus_queue.cc:237] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } }
I20260808 13:15:14.070935 20792 catalog_manager.cc:5495] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f5e80a1f281433d846956c9cdcac7a2 (127.31.59.130). New cstate: current_term: 1 leader_uuid: "0f5e80a1f281433d846956c9cdcac7a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f5e80a1f281433d846956c9cdcac7a2" member_type: VOTER last_known_addr { host: "127.31.59.130" port: 42067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 } health_report { overall_health: UNKNOWN } } }
I20260808 13:15:14.101564 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.096314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.31.59.130:42067)
0808 13:15:14.096537 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.096544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.096656 (+ 112us) server_negotiation.cc:407] Connection header received
0808 13:15:14.096880 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.096886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.097014 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.097176 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:14.097524 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.098238 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.099543 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.099947 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.100036 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.100447 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:14.100938 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:14.101042 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.101197 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.101269 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:15:14.110013 21233 raft_consensus.cc:1261] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 1 FOLLOWER]: Refusing update from remote peer 0f5e80a1f281433d846956c9cdcac7a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:14.110013 20948 raft_consensus.cc:1261] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 1 FOLLOWER]: Refusing update from remote peer 0f5e80a1f281433d846956c9cdcac7a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:14.111102 21314 consensus_queue.cc:1035] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59bebce78daf470b979de3ae363dc95b" member_type: VOTER last_known_addr { host: "127.31.59.131" port: 37581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:14.111404 21317 consensus_queue.cc:1035] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84edc32697e425d9d7e56249a5d390f" member_type: VOTER last_known_addr { host: "127.31.59.129" port: 36231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:15:14.112950 21284 tablet.cc:2369] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:14.124830 21321 mvcc.cc:205] Tried to move back new op lower bound from 7316254368183398400 to 7316254368020447232. Current Snapshot: MvccSnapshot[applied={T|T < 7316254368183398400}]
I20260808 13:15:14.139254 20792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:14.139559 20792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:15:14.143252 20792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d753bf003133421cb4e2a7e58129f363: Sending DeleteTablet for 3 replicas of tablet aac8851c82cc4d05ac36c928a60c6ab8
I20260808 13:15:14.144439 21213 tablet_service.cc:1514] Processing DeleteTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:14 UTC) from {username='slave'} at 127.0.0.1:56444
I20260808 13:15:14.144455 21068 tablet_service.cc:1514] Processing DeleteTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:14 UTC) from {username='slave'} at 127.0.0.1:43000
I20260808 13:15:14.144681 20927 tablet_service.cc:1514] Processing DeleteTablet for tablet aac8851c82cc4d05ac36c928a60c6ab8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:15:14 UTC) from {username='slave'} at 127.0.0.1:51950
I20260808 13:15:14.146297 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20861
I20260808 13:15:14.157936 21333 tablet_replica.cc:330] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: stopping tablet replica
I20260808 13:15:14.158324 21333 raft_consensus.cc:2229] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:14.158779 21333 raft_consensus.cc:2258] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:14.163389 21333 ts_tablet_manager.cc:1905] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:14.170897 21333 ts_tablet_manager.cc:1918] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:14.171167 21333 log.cc:1196] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-2/wal/wals/aac8851c82cc4d05ac36c928a60c6ab8
I20260808 13:15:14.171805 21333 ts_tablet_manager.cc:1939] T aac8851c82cc4d05ac36c928a60c6ab8 P 59bebce78daf470b979de3ae363dc95b: Deleting consensus metadata
I20260808 13:15:14.173875 20776 catalog_manager.cc:4841] TS 59bebce78daf470b979de3ae363dc95b (127.31.59.131:37581): tablet aac8851c82cc4d05ac36c928a60c6ab8 (table test-table [id=42329ef423a04bf99ef69478241fd394]) successfully deleted
I20260808 13:15:14.174953 21335 tablet_replica.cc:330] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: stopping tablet replica
I20260808 13:15:14.175478 21335 raft_consensus.cc:2229] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:15:14.176110 21335 raft_consensus.cc:2258] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:14.177045 21334 tablet_replica.cc:330] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: stopping tablet replica
I20260808 13:15:14.177577 21334 raft_consensus.cc:2229] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:15:14.178390 21334 raft_consensus.cc:2258] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:15:14.178789 21335 ts_tablet_manager.cc:1905] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:14.180188 21334 ts_tablet_manager.cc:1905] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:15:14.184440 21335 ts_tablet_manager.cc:1918] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:14.184644 21335 log.cc:1196] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-0/wal/wals/aac8851c82cc4d05ac36c928a60c6ab8
I20260808 13:15:14.185120 21335 ts_tablet_manager.cc:1939] T aac8851c82cc4d05ac36c928a60c6ab8 P e84edc32697e425d9d7e56249a5d390f: Deleting consensus metadata
I20260808 13:15:14.186810 20775 catalog_manager.cc:4841] TS e84edc32697e425d9d7e56249a5d390f (127.31.59.129:36231): tablet aac8851c82cc4d05ac36c928a60c6ab8 (table test-table [id=42329ef423a04bf99ef69478241fd394]) successfully deleted
I20260808 13:15:14.187772 21334 ts_tablet_manager.cc:1918] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:15:14.187956 21334 log.cc:1196] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Deleting WAL directory at /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194589892156-31982-0/minicluster-data/ts-1/wal/wals/aac8851c82cc4d05ac36c928a60c6ab8
I20260808 13:15:14.188598 21334 ts_tablet_manager.cc:1939] T aac8851c82cc4d05ac36c928a60c6ab8 P 0f5e80a1f281433d846956c9cdcac7a2: Deleting consensus metadata
I20260808 13:15:14.188777 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.147863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.31.59.129:36231)
0808 13:15:14.148051 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.148058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.171904 (+ 23846us) server_negotiation.cc:407] Connection header received
0808 13:15:14.172458 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.172465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.172613 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.172693 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:14.173529 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.174823 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.177115 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.177770 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.177873 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.187281 (+ 9408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:14.187290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:14.187303 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:14.187837 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:14.187892 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:14.187919 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:14.188060 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.188318 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.188438 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:15:14.189533 20990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:14.190529 20777 catalog_manager.cc:4841] TS 0f5e80a1f281433d846956c9cdcac7a2 (127.31.59.130:42067): tablet aac8851c82cc4d05ac36c928a60c6ab8 (table test-table [id=42329ef423a04bf99ef69478241fd394]) successfully deleted
I20260808 13:15:14.279896 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20861
I20260808 13:15:14.307096 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20999
I20260808 13:15:14.314819 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.307925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.31.59.130:42067)
0808 13:15:14.308109 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.308113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.308162 (+ 49us) server_negotiation.cc:407] Connection header received
0808 13:15:14.308348 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.308352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.308449 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.308533 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:14.308989 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.309854 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.311686 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.312116 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.312202 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.313381 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:14.313391 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:14.313406 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:14.314057 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:14.314109 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:14.314130 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:14.314210 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.314354 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.314432 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:15:14.315546 21134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:14.429740 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20999
I20260808 13:15:14.450697 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21142
I20260808 13:15:14.457252 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:14.451452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.31.59.131:37581)
0808 13:15:14.451634 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:14.451640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:14.451660 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:15:14.451864 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:14.451868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:14.451949 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:14.452034 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:14.452484 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.453210 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.454483 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:14.454870 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:14.454979 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:14.455986 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:14.455995 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:14.456010 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:14.456553 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:14.456604 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:14.456633 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:14.456729 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:14.456856 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 13:15:14.456966 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:15:14.458061 21277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:14.545092 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21142
I20260808 13:15:14.562551 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 20759
I20260808 13:15:14.563772 20828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:14.674319 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 20759
2026-08-08T13:15:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2853 ms)
[----------] 6 tests from EncryptionPolicyTest (22189 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:14 dist-test-slave-49h3 krb5kdc[21347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:14 dist-test-slave-49h3 krb5kdc[21347](info): set up 2 sockets
Aug 08 13:15:14 dist-test-slave-49h3 krb5kdc[21347](info): commencing operation
krb5kdc: starting...
W20260808 13:15:15.768782 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:15 dist-test-slave-49h3 krb5kdc[21347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:16.105059 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:16.446283 21361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:16.446583 21361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:16.446717 21361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:16.453974 21361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:16.454118 21361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:16.454197 21361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:16.454283 21361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:16.454344 21361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:16.464313 21361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:16.465737 21361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:16.467012 21361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:16.478142 21366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:16.478225 21367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:16.478256 21369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:16.479079 21361 server_base.cc:1034] running on GCE node
I20260808 13:15:16.479805 21361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:16.481297 21361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:16.482533 21361 hybrid_clock.cc:648] HybridClock initialized: now 1786194916482483 us; error 37 us; skew 500 ppm
I20260808 13:15:16.482908 21361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:16 dist-test-slave-49h3 krb5kdc[21347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:16.486235 21361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:16.487784 21361 webserver.cc:460] Webserver started at http://127.0.0.1:44739/ using document root <none> and password file <none>
I20260808 13:15:16.488468 21361 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:16.488600 21361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:16.488893 21361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:16.491981 21361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "b431b8c0ef38449391b4846d64d305c6"
format_stamp: "Formatted at 2026-08-08 13:15:16 on dist-test-slave-49h3"
server_key: "9f5e9f4c2a55b5ecfcda042761e4a0e7"
server_key_iv: "82d8d4186cc16c1b790217589316fd6d"
server_key_version: "encryptionkey@0"
I20260808 13:15:16.492776 21361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "b431b8c0ef38449391b4846d64d305c6"
format_stamp: "Formatted at 2026-08-08 13:15:16 on dist-test-slave-49h3"
server_key: "9f5e9f4c2a55b5ecfcda042761e4a0e7"
server_key_iv: "82d8d4186cc16c1b790217589316fd6d"
server_key_version: "encryptionkey@0"
I20260808 13:15:16.498018 21361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 13:15:16.501410 21376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:16.502768 21361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:16.502934 21361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "b431b8c0ef38449391b4846d64d305c6"
format_stamp: "Formatted at 2026-08-08 13:15:16 on dist-test-slave-49h3"
server_key: "9f5e9f4c2a55b5ecfcda042761e4a0e7"
server_key_iv: "82d8d4186cc16c1b790217589316fd6d"
server_key_version: "encryptionkey@0"
I20260808 13:15:16.503115 21361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:16.533420 21361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:16.538039 21361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:16.538345 21361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:16.556099 21361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46161
I20260808 13:15:16.556162 21431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46161 every 8 connection(s)
I20260808 13:15:16.557518 21361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:16.561234 21433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:16.566766 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21361
I20260808 13:15:16.566972 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:16.567621 31982 external_mini_cluster.cc:1428] Setting key b574b566007f9fc6d6f02e0d4bce8acd
I20260808 13:15:16.572317 21433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6: Bootstrap starting.
I20260808 13:15:16.575413 21433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:16.576460 21433 log.cc:824] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:16.579881 21433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6: No bootstrap required, opened a new log
Aug 08 13:15:16 dist-test-slave-49h3 krb5kdc[21347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:15:16.589704 21433 raft_consensus.cc:348] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260808 13:15:16.590188 21433 raft_consensus.cc:374] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:16.590306 21433 raft_consensus.cc:729] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b431b8c0ef38449391b4846d64d305c6, State: Initialized, Role: FOLLOWER
I20260808 13:15:16.591024 21433 consensus_queue.cc:260] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260808 13:15:16.591331 21433 raft_consensus.cc:388] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:16.591481 21433 raft_consensus.cc:482] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:16.591689 21433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:16.592574 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:16.569640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33259 (local address 127.0.0.1:46161)
0808 13:15:16.570503 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:16.570515 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:16.570554 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:15:16.572486 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:16.572513 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:16.572921 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:16.573289 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:16.574843 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:16.576221 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:16.577864 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:16.578932 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:16.579148 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:16.583509 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:16.583520 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:16.583541 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:16.583593 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:16.587709 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:16.588515 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:16.588524 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:16.588533 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:16.588773 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:16.589175 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:16.589179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:16.589182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:16.589943 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:16.590280 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:16.590740 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 13:15:16.591095 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":403,"threads_started":1}
I20260808 13:15:16.594434 21433 raft_consensus.cc:504] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260808 13:15:16.594909 21433 leader_election.cc:302] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b431b8c0ef38449391b4846d64d305c6; no voters:
I20260808 13:15:16.595562 21433 leader_election.cc:288] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:16.595649 21438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:16.596289 21438 raft_consensus.cc:686] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [term 1 LEADER]: Becoming Leader. State: Replica: b431b8c0ef38449391b4846d64d305c6, State: Running, Role: LEADER
I20260808 13:15:16.596872 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:16.596861 21438 consensus_queue.cc:237] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260808 13:15:16.598453 21433 sys_catalog.cc:564] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:16.602039 21440 sys_catalog.cc:455] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b431b8c0ef38449391b4846d64d305c6. Latest consensus state: current_term: 1 leader_uuid: "b431b8c0ef38449391b4846d64d305c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260808 13:15:16.602451 21440 sys_catalog.cc:458] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:16.603343 21441 sys_catalog.cc:455] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b431b8c0ef38449391b4846d64d305c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b431b8c0ef38449391b4846d64d305c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260808 13:15:16.603574 21450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:16.603626 21441 sys_catalog.cc:458] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:16.608572 21450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:16.626470 21450 catalog_manager.cc:1347] Generated new cluster ID: 7f3c4736161c458fa467abeb87dc3e6b
I20260808 13:15:16.626612 21450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:16.638384 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:16.614541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56019 (local address 127.0.0.1:46161)
0808 13:15:16.614728 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:16.614735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:16.620092 (+ 5357us) server_negotiation.cc:407] Connection header received
0808 13:15:16.620292 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:16.620298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:16.620384 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:16.620500 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:16.622077 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:16.623606 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:16.625577 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:16.626344 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:16.626446 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:16.630078 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:16.630086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:16.630095 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:16.630138 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:16.634196 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:16.635322 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:16.635328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:16.635333 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:16.635520 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:16.636123 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:16.636129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:16.636133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:16.636582 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:16.636926 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:16.637718 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 13:15:16.637948 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:15:16.658394 21450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:16.659811 21450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:16.669194 21450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b431b8c0ef38449391b4846d64d305c6: Generated new TSK 0
I20260808 13:15:16.670125 21450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:16.681948 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21361
I20260808 13:15:16.683393 21427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:16.776801 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21361
W20260808 13:15:16.795449 21344 connection.cc:535] client connection to 127.0.0.1:46161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46161): Connection reset by peer (error 104)
2026-08-08T13:15:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2096 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:16 dist-test-slave-49h3 krb5kdc[21472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:16 dist-test-slave-49h3 krb5kdc[21472](info): set up 2 sockets
Aug 08 13:15:16 dist-test-slave-49h3 krb5kdc[21472](info): commencing operation
krb5kdc: starting...
W20260808 13:15:17.864836 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:18 dist-test-slave-49h3 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:18.196624 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:18.538491 21485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:18.538802 21485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:18.538955 21485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:18.546446 21485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:18.546571 21485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:18.546656 21485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:18.546751 21485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:18.546808 21485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:18.557003 21485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:18.558368 21485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:18.559705 21485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:18.571203 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:18.571146 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:18.571230 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:18.571981 21485 server_base.cc:1034] running on GCE node
I20260808 13:15:18.572744 21485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:18.574205 21485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:18.575402 21485 hybrid_clock.cc:648] HybridClock initialized: now 1786194918575353 us; error 37 us; skew 500 ppm
I20260808 13:15:18.575738 21485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:18 dist-test-slave-49h3 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:18.578800 21485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:18.580209 21485 webserver.cc:460] Webserver started at http://127.0.0.1:36617/ using document root <none> and password file <none>
I20260808 13:15:18.580878 21485 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:18.581001 21485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:18.581288 21485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:18.584435 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "63c90ad9edc7436ba9a9ef980c0d685c"
format_stamp: "Formatted at 2026-08-08 13:15:18 on dist-test-slave-49h3"
server_key: "2e30cdf8f20f82b76a1ad6e47945eb3d"
server_key_iv: "164179bcdd87bbc61b5f62077798497c"
server_key_version: "encryptionkey@0"
I20260808 13:15:18.585218 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "63c90ad9edc7436ba9a9ef980c0d685c"
format_stamp: "Formatted at 2026-08-08 13:15:18 on dist-test-slave-49h3"
server_key: "2e30cdf8f20f82b76a1ad6e47945eb3d"
server_key_iv: "164179bcdd87bbc61b5f62077798497c"
server_key_version: "encryptionkey@0"
I20260808 13:15:18.590402 21485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:15:18.593786 21501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:18.595024 21485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:15:18.595180 21485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "63c90ad9edc7436ba9a9ef980c0d685c"
format_stamp: "Formatted at 2026-08-08 13:15:18 on dist-test-slave-49h3"
server_key: "2e30cdf8f20f82b76a1ad6e47945eb3d"
server_key_iv: "164179bcdd87bbc61b5f62077798497c"
server_key_version: "encryptionkey@0"
I20260808 13:15:18.595353 21485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:18.620272 21485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:18.624624 21485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:18.624918 21485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:18.643049 21485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34489
I20260808 13:15:18.643072 21555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34489 every 8 connection(s)
I20260808 13:15:18.644476 21485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:18.648212 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:18.648258 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21485
I20260808 13:15:18.648422 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:18.648887 31982 external_mini_cluster.cc:1428] Setting key 041ae7d2d825a89d4030fcce536fc117
I20260808 13:15:18.661263 21556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c: Bootstrap starting.
I20260808 13:15:18.662890 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:18.650700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46071 (local address 127.0.0.1:34489)
0808 13:15:18.651550 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:18.651562 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:18.651601 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:15:18.653999 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:18.654022 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:18.654423 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:18.654757 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:18.655339 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:18.656574 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:18.658407 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:18.659121 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:18.659252 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:18.660538 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:18.660546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:18.660559 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:18.661098 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:18.661178 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:18.661209 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:18.661326 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:18.661646 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 13:15:18.661868 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":387,"threads_started":1}
I20260808 13:15:18.664577 21556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:18.665778 21556 log.cc:824] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:18.666605 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:18.668468 21556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c: No bootstrap required, opened a new log
I20260808 13:15:18.677755 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:18.671002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54357 (local address 127.0.0.1:34489)
0808 13:15:18.671185 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:18.671191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:18.671736 (+ 545us) server_negotiation.cc:407] Connection header received
0808 13:15:18.671921 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:18.671925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:18.671987 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:18.672070 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:18.672644 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:18.673416 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:18.675308 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:18.675792 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:18.675891 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:18.677022 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:18.677027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:18.677030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:18.677048 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:18.677102 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:18.677117 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:18.677197 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:18.677368 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 13:15:18.677413 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:15:18.677942 21556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260808 13:15:18.678381 21556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:18.678479 21556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c90ad9edc7436ba9a9ef980c0d685c, State: Initialized, Role: FOLLOWER
I20260808 13:15:18.679080 21556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260808 13:15:18.679322 21556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:18.679466 21556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:18.679613 21556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:18.681953 21556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260808 13:15:18.682448 21556 leader_election.cc:302] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63c90ad9edc7436ba9a9ef980c0d685c; no voters:
I20260808 13:15:18.683041 21556 leader_election.cc:288] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:18.683176 21566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:18.684129 21566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [term 1 LEADER]: Becoming Leader. State: Replica: 63c90ad9edc7436ba9a9ef980c0d685c, State: Running, Role: LEADER
I20260808 13:15:18.684620 21566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260808 13:15:18.685115 21556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:18.687698 21567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260808 13:15:18.687826 21568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c90ad9edc7436ba9a9ef980c0d685c. Latest consensus state: current_term: 1 leader_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c90ad9edc7436ba9a9ef980c0d685c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260808 13:15:18.688175 21567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:18.688205 21568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:18.689827 21575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:18.694450 21575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:18.703368 21575 catalog_manager.cc:1347] Generated new cluster ID: 2c60e880139b485d85b8cb4f7cf9b39f
I20260808 13:15:18.703490 21575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:18.724516 21575 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:18.726011 21575 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:18.732252 21575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63c90ad9edc7436ba9a9ef980c0d685c: Generated new TSK 0
I20260808 13:15:18.733057 21575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:18.742228 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21485
W20260808 13:15:18.743546 21550 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46071
W20260808 13:15:18.744380 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21485: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:18.744586 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21485
W20260808 13:15:18.763840 21467 connection.cc:535] client connection to 127.0.0.1:34489 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34489): Connection reset by peer (error 104)
2026-08-08T13:15:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1968 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:18 dist-test-slave-49h3 krb5kdc[21589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:18 dist-test-slave-49h3 krb5kdc[21589](info): set up 2 sockets
Aug 08 13:15:18 dist-test-slave-49h3 krb5kdc[21589](info): commencing operation
krb5kdc: starting...
W20260808 13:15:19.845183 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.004s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:20 dist-test-slave-49h3 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:20.168401 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:20.504829 21603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:20.505127 21603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:20.505227 21603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:20.512454 21603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:20.512571 21603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:20.512627 21603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:20.512701 21603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:20.512746 21603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:20.522738 21603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:20.523988 21603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:20.525352 21603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:20.536553 21608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:20.536592 21611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:20.536612 21609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:20.537652 21603 server_base.cc:1034] running on GCE node
I20260808 13:15:20.538376 21603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:20.539701 21603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:20.540902 21603 hybrid_clock.cc:648] HybridClock initialized: now 1786194920540849 us; error 39 us; skew 500 ppm
I20260808 13:15:20.541249 21603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:20 dist-test-slave-49h3 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194920, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:20.544344 21603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:20.545854 21603 webserver.cc:460] Webserver started at http://127.0.0.1:41529/ using document root <none> and password file <none>
I20260808 13:15:20.546545 21603 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:20.546664 21603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:20.546954 21603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:20.549976 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "883b3466d6fd41048700e3fb413ca18a"
format_stamp: "Formatted at 2026-08-08 13:15:20 on dist-test-slave-49h3"
server_key: "32471371f33928205eb0a45d727d84ce"
server_key_iv: "4c9595948b69805ef536f448255edf90"
server_key_version: "encryptionkey@0"
I20260808 13:15:20.550743 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "883b3466d6fd41048700e3fb413ca18a"
format_stamp: "Formatted at 2026-08-08 13:15:20 on dist-test-slave-49h3"
server_key: "32471371f33928205eb0a45d727d84ce"
server_key_iv: "4c9595948b69805ef536f448255edf90"
server_key_version: "encryptionkey@0"
I20260808 13:15:20.555742 21603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:20.559051 21620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:20.560263 21603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:15:20.560421 21603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "883b3466d6fd41048700e3fb413ca18a"
format_stamp: "Formatted at 2026-08-08 13:15:20 on dist-test-slave-49h3"
server_key: "32471371f33928205eb0a45d727d84ce"
server_key_iv: "4c9595948b69805ef536f448255edf90"
server_key_version: "encryptionkey@0"
I20260808 13:15:20.560593 21603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:20.566008 21603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:20.569686 21603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:20.569960 21603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:20.587281 21603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41563
I20260808 13:15:20.587312 21677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41563 every 8 connection(s)
I20260808 13:15:20.588657 21603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:20.589325 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21603
I20260808 13:15:20.589540 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:20.590103 31982 external_mini_cluster.cc:1428] Setting key 186d395bd913020a749a8e775857aee4
I20260808 13:15:20.592907 21678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:20.597649 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:20.591842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42467 (local address 127.0.0.1:41563)
0808 13:15:20.592699 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:20.592711 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:20.592754 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:15:20.594419 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:20.594441 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:20.594736 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:20.595074 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:20.596068 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:20.596074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:20.596086 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:20.596100 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:20.596182 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:20.596211 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:20.596426 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:20.596555 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 13:15:20.596729 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260808 13:15:20.601248 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:20.604205 21678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a: Bootstrap starting.
I20260808 13:15:20.607605 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:20.605406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34337 (local address 127.0.0.1:41563)
0808 13:15:20.605609 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:20.605614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:20.605900 (+ 286us) server_negotiation.cc:407] Connection header received
0808 13:15:20.606021 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:20.606026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:20.606097 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:20.606208 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:20.607023 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:20.607029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:20.607034 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:20.607051 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:20.607098 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:20.607117 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:20.607235 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:20.607280 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 13:15:20.607332 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:15:20.607703 21678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:20.608985 21678 log.cc:824] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:20.611445 21678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a: No bootstrap required, opened a new log
I20260808 13:15:20.618835 21678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } }
I20260808 13:15:20.619212 21678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:20.619294 21678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 883b3466d6fd41048700e3fb413ca18a, State: Initialized, Role: FOLLOWER
I20260808 13:15:20.619916 21678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } }
I20260808 13:15:20.620199 21678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:20.620368 21678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:20.620561 21678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:20.622851 21678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } }
I20260808 13:15:20.623397 21678 leader_election.cc:302] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 883b3466d6fd41048700e3fb413ca18a; no voters:
I20260808 13:15:20.623934 21678 leader_election.cc:288] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:20.624075 21688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:20.624696 21688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [term 1 LEADER]: Becoming Leader. State: Replica: 883b3466d6fd41048700e3fb413ca18a, State: Running, Role: LEADER
I20260808 13:15:20.625226 21688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } }
I20260808 13:15:20.626116 21678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:20.627903 21689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "883b3466d6fd41048700e3fb413ca18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } } }
I20260808 13:15:20.628392 21689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:20.628242 21690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 883b3466d6fd41048700e3fb413ca18a. Latest consensus state: current_term: 1 leader_uuid: "883b3466d6fd41048700e3fb413ca18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883b3466d6fd41048700e3fb413ca18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41563 } } }
I20260808 13:15:20.628859 21690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:20.629706 21697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:20.639290 21697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:20.648151 21697 catalog_manager.cc:1347] Generated new cluster ID: 5f2b48c0e01f41489105272083e5d37a
I20260808 13:15:20.648280 21697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:20.669552 21697 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:20.670059 21697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:20.686151 21697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 883b3466d6fd41048700e3fb413ca18a: Generated new TSK 0
I20260808 13:15:20.687216 21697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:20.694689 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21603
W20260808 13:15:20.695678 21673 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42467
W20260808 13:15:20.696398 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21603: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:20.696552 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21603
2026-08-08T13:15:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1951 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:20 dist-test-slave-49h3 krb5kdc[21711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:20 dist-test-slave-49h3 krb5kdc[21711](info): set up 2 sockets
Aug 08 13:15:20 dist-test-slave-49h3 krb5kdc[21711](info): commencing operation
krb5kdc: starting...
W20260808 13:15:21.785320 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:21 dist-test-slave-49h3 krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:22.110926 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:22.447227 21724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:22.447515 21724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:22.447659 21724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:22.454885 21724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:22.455008 21724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:22.455098 21724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:22.455186 21724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:22.455241 21724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:22.465160 21724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:22.466529 21724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:22.467844 21724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:22.479190 21729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:22.479154 21733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:22.479081 21730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:22.480170 21724 server_base.cc:1034] running on GCE node
I20260808 13:15:22.480857 21724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:22.482205 21724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:22.483397 21724 hybrid_clock.cc:648] HybridClock initialized: now 1786194922483357 us; error 27 us; skew 500 ppm
I20260808 13:15:22.483713 21724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:22 dist-test-slave-49h3 krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:22.486752 21724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:22.488191 21724 webserver.cc:460] Webserver started at http://127.0.0.1:44323/ using document root <none> and password file <none>
I20260808 13:15:22.488839 21724 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:22.488961 21724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:22.489295 21724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:22.492358 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "17c7190006b047b98f9396e604bc2f16"
format_stamp: "Formatted at 2026-08-08 13:15:22 on dist-test-slave-49h3"
server_key: "3c90f296d892f190128833196a79c2a8"
server_key_iv: "801cf95955e6cfdfc971b890d709662b"
server_key_version: "encryptionkey@0"
I20260808 13:15:22.493117 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "17c7190006b047b98f9396e604bc2f16"
format_stamp: "Formatted at 2026-08-08 13:15:22 on dist-test-slave-49h3"
server_key: "3c90f296d892f190128833196a79c2a8"
server_key_iv: "801cf95955e6cfdfc971b890d709662b"
server_key_version: "encryptionkey@0"
I20260808 13:15:22.498217 21724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 13:15:22.501669 21740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:22.502983 21724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:15:22.503189 21724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "17c7190006b047b98f9396e604bc2f16"
format_stamp: "Formatted at 2026-08-08 13:15:22 on dist-test-slave-49h3"
server_key: "3c90f296d892f190128833196a79c2a8"
server_key_iv: "801cf95955e6cfdfc971b890d709662b"
server_key_version: "encryptionkey@0"
I20260808 13:15:22.503378 21724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:22.539820 21724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:22.543919 21724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:22.544214 21724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:22.561865 21724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42497
I20260808 13:15:22.561916 21795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42497 every 8 connection(s)
I20260808 13:15:22.563241 21724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:22.566838 21796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:22.571759 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21724
I20260808 13:15:22.571983 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:22.572561 31982 external_mini_cluster.cc:1428] Setting key 16bad8bcf2b8dbba38a219334053e882
I20260808 13:15:22.578127 21796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16: Bootstrap starting.
I20260808 13:15:22.581164 21796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:22.582204 21796 log.cc:824] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:22.584942 21796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16: No bootstrap required, opened a new log
Aug 08 13:15:22 dist-test-slave-49h3 krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:15:22.592767 21796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } }
I20260808 13:15:22.593170 21796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:22.593271 21796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c7190006b047b98f9396e604bc2f16, State: Initialized, Role: FOLLOWER
I20260808 13:15:22.594002 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:22.574409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34607 (local address 127.0.0.1:42497)
0808 13:15:22.575208 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:22.575219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:22.575265 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:15:22.577025 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:22.577048 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:22.577415 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:22.577793 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:22.579283 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:22.580646 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:22.581982 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:22.582738 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:22.582947 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:22.586867 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:22.586875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:22.586888 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:22.586928 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:22.589994 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:22.590644 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:22.590650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:22.590656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:22.590816 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:22.591208 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:22.591211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:22.591213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:22.591765 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:22.592065 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:22.592506 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 13:15:22.592899 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":357,"threads_started":1}
I20260808 13:15:22.593886 21796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } }
I20260808 13:15:22.594163 21796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:22.594318 21796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:22.594480 21796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:22.596941 21796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } }
I20260808 13:15:22.597484 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:22.597430 21796 leader_election.cc:302] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17c7190006b047b98f9396e604bc2f16; no voters:
I20260808 13:15:22.598057 21796 leader_election.cc:288] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:22.598192 21802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:22.599848 21796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:22.606557 21802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [term 1 LEADER]: Becoming Leader. State: Replica: 17c7190006b047b98f9396e604bc2f16, State: Running, Role: LEADER
I20260808 13:15:22.607246 21802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } }
I20260808 13:15:22.610862 21804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17c7190006b047b98f9396e604bc2f16. Latest consensus state: current_term: 1 leader_uuid: "17c7190006b047b98f9396e604bc2f16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } } }
I20260808 13:15:22.611277 21804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:22.611904 21803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17c7190006b047b98f9396e604bc2f16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c7190006b047b98f9396e604bc2f16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42497 } } }
I20260808 13:15:22.612193 21803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:22.625849 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:22.608246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60219 (local address 127.0.0.1:42497)
0808 13:15:22.608498 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:22.608505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:22.609381 (+ 876us) server_negotiation.cc:407] Connection header received
0808 13:15:22.609613 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:22.609622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:22.609711 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:22.609808 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:22.611902 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:22.613026 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:22.614907 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:22.615610 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:22.615709 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:22.618987 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:22.618994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:22.618999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:22.619039 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:22.622511 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:22.623388 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:22.623394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:22.623398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:22.623544 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:22.623969 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:22.623974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:22.623978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:22.624391 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:22.624615 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:22.625188 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 13:15:22.625422 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 13:15:22.633498 21823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:22.638924 21823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:22.648319 21823 catalog_manager.cc:1347] Generated new cluster ID: 779bbb08291643b69ae60d503a2d5b74
I20260808 13:15:22.648442 21823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:22.671821 21823 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:22.673223 21823 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:22.679610 21823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17c7190006b047b98f9396e604bc2f16: Generated new TSK 0
I20260808 13:15:22.680348 21823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:22.685797 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21724
I20260808 13:15:22.686872 21790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:22.778544 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21724
2026-08-08T13:15:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2084 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:22 dist-test-slave-49h3 krb5kdc[21836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:22 dist-test-slave-49h3 krb5kdc[21836](info): set up 2 sockets
Aug 08 13:15:22 dist-test-slave-49h3 krb5kdc[21836](info): commencing operation
krb5kdc: starting...
W20260808 13:15:23.876621 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:24 dist-test-slave-49h3 krb5kdc[21836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:24.205034 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:24.539772 21850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:24.540042 21850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:24.540190 21850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:24.547380 21850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:24.547497 21850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:24.547575 21850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:24.547655 21850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:24.547712 21850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:24.557662 21850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:24.559036 21850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:24.560354 21850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:24.571676 21859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:24.571774 21856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:24.572254 21855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:24.572918 21850 server_base.cc:1034] running on GCE node
I20260808 13:15:24.573695 21850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:24.575052 21850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:24.576246 21850 hybrid_clock.cc:648] HybridClock initialized: now 1786194924576207 us; error 24 us; skew 500 ppm
I20260808 13:15:24.576542 21850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:24 dist-test-slave-49h3 krb5kdc[21836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194924, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:24.579416 21850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:24.580816 21850 webserver.cc:460] Webserver started at http://127.0.0.1:45111/ using document root <none> and password file <none>
I20260808 13:15:24.581426 21850 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:24.581576 21850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:24.581833 21850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:24.584664 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "0e1a32108fdc4fc89409a132b6fd2a11"
format_stamp: "Formatted at 2026-08-08 13:15:24 on dist-test-slave-49h3"
server_key: "1e3c9fa6e07daa5f709aff6da7b5389a"
server_key_iv: "02652b393909c2e2a01e8a65f9bcda67"
server_key_version: "encryptionkey@0"
I20260808 13:15:24.585386 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "0e1a32108fdc4fc89409a132b6fd2a11"
format_stamp: "Formatted at 2026-08-08 13:15:24 on dist-test-slave-49h3"
server_key: "1e3c9fa6e07daa5f709aff6da7b5389a"
server_key_iv: "02652b393909c2e2a01e8a65f9bcda67"
server_key_version: "encryptionkey@0"
I20260808 13:15:24.590287 21850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:15:24.593565 21867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:24.594786 21850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:15:24.594925 21850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "0e1a32108fdc4fc89409a132b6fd2a11"
format_stamp: "Formatted at 2026-08-08 13:15:24 on dist-test-slave-49h3"
server_key: "1e3c9fa6e07daa5f709aff6da7b5389a"
server_key_iv: "02652b393909c2e2a01e8a65f9bcda67"
server_key_version: "encryptionkey@0"
I20260808 13:15:24.595084 21850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:24.610307 21850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:24.614199 21850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:24.614457 21850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:24.632426 21850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46185
I20260808 13:15:24.632457 21923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46185 every 8 connection(s)
I20260808 13:15:24.633894 21850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:24.636003 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21850
I20260808 13:15:24.636173 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:24.636613 31982 external_mini_cluster.cc:1428] Setting key 3416b58cca5780755ab0d5478d9f12b0
I20260808 13:15:24.637753 21924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:24.651091 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:24.638273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41749 (local address 127.0.0.1:46185)
0808 13:15:24.639025 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:24.639040 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:24.639090 (+ 50us) server_negotiation.cc:407] Connection header received
0808 13:15:24.641021 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:24.641046 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:24.641579 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:24.642001 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:24.642513 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:24.644551 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:24.645994 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:24.647034 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:24.647237 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:24.648315 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:24.648330 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:24.648349 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:24.649083 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:24.649189 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:24.649225 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:24.649349 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:24.649510 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 13:15:24.649793 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":312,"threads_started":1}
I20260808 13:15:24.652330 21924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11: Bootstrap starting.
I20260808 13:15:24.654904 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:24.655999 21924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:24.657186 21924 log.cc:824] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:24.660468 21924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11: No bootstrap required, opened a new log
I20260808 13:15:24.665906 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:24.659981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54097 (local address 127.0.0.1:46185)
0808 13:15:24.660178 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:24.660184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:24.660623 (+ 439us) server_negotiation.cc:407] Connection header received
0808 13:15:24.660787 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:24.660792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:24.660855 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:24.660934 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:24.661484 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:24.662223 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:24.663558 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:24.664034 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:24.664154 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:24.665184 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:24.665188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:24.665195 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:24.665211 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:24.665252 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:24.665268 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:24.665348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:24.665499 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 13:15:24.665594 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 13:15:24.668572 21924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } }
I20260808 13:15:24.668933 21924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:24.669028 21924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1a32108fdc4fc89409a132b6fd2a11, State: Initialized, Role: FOLLOWER
I20260808 13:15:24.669627 21924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } }
I20260808 13:15:24.669858 21924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:24.669983 21924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:24.670152 21924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:24.672336 21924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } }
I20260808 13:15:24.672822 21924 leader_election.cc:302] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e1a32108fdc4fc89409a132b6fd2a11; no voters:
I20260808 13:15:24.673465 21924 leader_election.cc:288] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:24.673555 21936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:24.674191 21936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [term 1 LEADER]: Becoming Leader. State: Replica: 0e1a32108fdc4fc89409a132b6fd2a11, State: Running, Role: LEADER
I20260808 13:15:24.674760 21936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } }
I20260808 13:15:24.675516 21924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:24.677276 21937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } } }
I20260808 13:15:24.677795 21937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:24.677732 21938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e1a32108fdc4fc89409a132b6fd2a11. Latest consensus state: current_term: 1 leader_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1a32108fdc4fc89409a132b6fd2a11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46185 } } }
I20260808 13:15:24.678148 21938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:24.679416 21944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:24.685107 21944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:24.693279 21944 catalog_manager.cc:1347] Generated new cluster ID: 92da0fa0709442e4bdeb1a1e8fd45dea
I20260808 13:15:24.693377 21944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:24.703174 21944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:24.705051 21944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:24.712073 21944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e1a32108fdc4fc89409a132b6fd2a11: Generated new TSK 0
I20260808 13:15:24.713114 21944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:24.721882 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21850
W20260808 13:15:24.723565 21918 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41749
W20260808 13:15:24.724337 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21850: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:24.724505 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21850
2026-08-08T13:15:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1943 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:24 dist-test-slave-49h3 krb5kdc[21958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:24 dist-test-slave-49h3 krb5kdc[21958](info): set up 2 sockets
Aug 08 13:15:24 dist-test-slave-49h3 krb5kdc[21958](info): commencing operation
krb5kdc: starting...
W20260808 13:15:25.820680 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:26 dist-test-slave-49h3 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:26.150965 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:26.492504 21976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:26.492780 21976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:26.492916 21976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:26.500078 21976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:26.500182 21976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:26.500264 21976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:26.500349 21976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:26.500408 21976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:26.510438 21976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:26.511754 21976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:26.513121 21976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:26.524479 21982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:26.524449 21983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:26.524706 21985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:26.525352 21976 server_base.cc:1034] running on GCE node
I20260808 13:15:26.526131 21976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:26.527499 21976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:26.528715 21976 hybrid_clock.cc:648] HybridClock initialized: now 1786194926528665 us; error 37 us; skew 500 ppm
I20260808 13:15:26.529047 21976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:26 dist-test-slave-49h3 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194926, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:26.538244 21976 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:26.539796 21976 webserver.cc:460] Webserver started at http://127.0.0.1:42329/ using document root <none> and password file <none>
I20260808 13:15:26.540475 21976 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:26.540601 21976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:26.540899 21976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:26.543948 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "3f1f95c0bac44c05a23ef8e49d65d86d"
format_stamp: "Formatted at 2026-08-08 13:15:26 on dist-test-slave-49h3"
server_key: "e6ea50150c5bf9a4b836a60087b5df27"
server_key_iv: "471857823f57211f2db15b68b0be4c4f"
server_key_version: "encryptionkey@0"
I20260808 13:15:26.544730 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "3f1f95c0bac44c05a23ef8e49d65d86d"
format_stamp: "Formatted at 2026-08-08 13:15:26 on dist-test-slave-49h3"
server_key: "e6ea50150c5bf9a4b836a60087b5df27"
server_key_iv: "471857823f57211f2db15b68b0be4c4f"
server_key_version: "encryptionkey@0"
I20260808 13:15:26.550011 21976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 13:15:26.553579 21993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:26.554904 21976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:26.555076 21976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "3f1f95c0bac44c05a23ef8e49d65d86d"
format_stamp: "Formatted at 2026-08-08 13:15:26 on dist-test-slave-49h3"
server_key: "e6ea50150c5bf9a4b836a60087b5df27"
server_key_iv: "471857823f57211f2db15b68b0be4c4f"
server_key_version: "encryptionkey@0"
I20260808 13:15:26.555274 21976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:26.561126 21976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:26.565131 21976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:26.565414 21976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:26.583305 21976 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35603
I20260808 13:15:26.583354 22050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35603 every 8 connection(s)
I20260808 13:15:26.584692 21976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:26.588296 22051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:26.591897 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 21976
I20260808 13:15:26.592072 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:26.592512 31982 external_mini_cluster.cc:1428] Setting key ccc07a3f2671d38e921c8c2aad9ff50d
I20260808 13:15:26.600137 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:26.594218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37461 (local address 127.0.0.1:35603)
0808 13:15:26.595030 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:26.595041 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:26.595083 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:15:26.596729 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:26.596752 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:26.597110 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:26.597507 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:26.598543 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:26.598550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:26.598566 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:26.598581 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:26.598641 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:26.598661 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:26.598881 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:26.599005 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 13:15:26.599186 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":341,"threads_started":1}
I20260808 13:15:26.600397 22051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d: Bootstrap starting.
I20260808 13:15:26.603888 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:26.603917 22051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:26.604880 22051 log.cc:824] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:26.607599 22051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d: No bootstrap required, opened a new log
I20260808 13:15:26.610608 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:26.608099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45091 (local address 127.0.0.1:35603)
0808 13:15:26.608296 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:26.608303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:26.608740 (+ 437us) server_negotiation.cc:407] Connection header received
0808 13:15:26.608911 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:26.608915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:26.608959 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:26.609036 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:26.609958 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:26.609962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:26.609965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:26.609983 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:26.610021 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:26.610035 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:26.610129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:26.610235 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 13:15:26.610284 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:15:26.615603 22051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } }
I20260808 13:15:26.615991 22051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:26.616078 22051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1f95c0bac44c05a23ef8e49d65d86d, State: Initialized, Role: FOLLOWER
I20260808 13:15:26.616611 22051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } }
I20260808 13:15:26.616848 22051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:26.616973 22051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:26.617120 22051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:26.619274 22051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } }
I20260808 13:15:26.619729 22051 leader_election.cc:302] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f1f95c0bac44c05a23ef8e49d65d86d; no voters:
I20260808 13:15:26.620306 22051 leader_election.cc:288] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:26.620419 22063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:26.621035 22063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [term 1 LEADER]: Becoming Leader. State: Replica: 3f1f95c0bac44c05a23ef8e49d65d86d, State: Running, Role: LEADER
I20260808 13:15:26.621542 22063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } }
I20260808 13:15:26.622243 22051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:26.624472 22064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } } }
I20260808 13:15:26.624485 22065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f1f95c0bac44c05a23ef8e49d65d86d. Latest consensus state: current_term: 1 leader_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1f95c0bac44c05a23ef8e49d65d86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35603 } } }
I20260808 13:15:26.624950 22065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:26.624950 22064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:26.631745 22069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:26.637363 22069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:26.645969 22069 catalog_manager.cc:1347] Generated new cluster ID: 16fe41d7d4e148a3a82a47c1b69bec62
I20260808 13:15:26.646135 22069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:26.662289 22069 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:26.662693 22069 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:26.671238 22069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f1f95c0bac44c05a23ef8e49d65d86d: Generated new TSK 0
I20260808 13:15:26.672025 22069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:26.678921 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21976
W20260808 13:15:26.680330 22046 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37461
W20260808 13:15:26.681049 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 21976: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:26.681186 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 21976
2026-08-08T13:15:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1955 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:26 dist-test-slave-49h3 krb5kdc[22086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:26 dist-test-slave-49h3 krb5kdc[22086](info): set up 2 sockets
Aug 08 13:15:26 dist-test-slave-49h3 krb5kdc[22086](info): commencing operation
krb5kdc: starting...
W20260808 13:15:27.777324 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:27 dist-test-slave-49h3 krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:28.112399 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:28.449854 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:28.450151 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:28.450281 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:28.457693 22101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:28.457813 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:28.457893 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:28.457980 22101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:28.458037 22101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:28.468084 22101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:28.469367 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:28.470654 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:28.482015 22106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:28.481948 22107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:28.481986 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:28.482746 22101 server_base.cc:1034] running on GCE node
I20260808 13:15:28.483546 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:28.484891 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:28.486146 22101 hybrid_clock.cc:648] HybridClock initialized: now 1786194928486093 us; error 40 us; skew 500 ppm
I20260808 13:15:28.486478 22101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:28 dist-test-slave-49h3 krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194928, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:28.489529 22101 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:28.490964 22101 webserver.cc:460] Webserver started at http://127.0.0.1:34057/ using document root <none> and password file <none>
I20260808 13:15:28.491638 22101 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:28.491755 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:28.492026 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:28.494987 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "c16e81583f4c4332a70d27c618fc3341"
format_stamp: "Formatted at 2026-08-08 13:15:28 on dist-test-slave-49h3"
server_key: "d22c50646f83e8b182645ea49ccbcd63"
server_key_iv: "348caaf05fefb2d31215a85ab5c86086"
server_key_version: "encryptionkey@0"
I20260808 13:15:28.495729 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "c16e81583f4c4332a70d27c618fc3341"
format_stamp: "Formatted at 2026-08-08 13:15:28 on dist-test-slave-49h3"
server_key: "d22c50646f83e8b182645ea49ccbcd63"
server_key_iv: "348caaf05fefb2d31215a85ab5c86086"
server_key_version: "encryptionkey@0"
I20260808 13:15:28.500604 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:15:28.503918 22117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:28.505120 22101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:28.505277 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "c16e81583f4c4332a70d27c618fc3341"
format_stamp: "Formatted at 2026-08-08 13:15:28 on dist-test-slave-49h3"
server_key: "d22c50646f83e8b182645ea49ccbcd63"
server_key_iv: "348caaf05fefb2d31215a85ab5c86086"
server_key_version: "encryptionkey@0"
I20260808 13:15:28.505476 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:28.521349 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:28.525625 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:28.525900 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:28.543692 22101 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38713
I20260808 13:15:28.543730 22173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38713 every 8 connection(s)
I20260808 13:15:28.544987 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:28.548638 22174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:28.554456 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22101
I20260808 13:15:28.554643 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:28.555080 31982 external_mini_cluster.cc:1428] Setting key f8067a4e45a9c29ba84e748eb6e1e749
I20260808 13:15:28.559352 22174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341: Bootstrap starting.
I20260808 13:15:28.562860 22174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:28.564006 22174 log.cc:824] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:28.567003 22174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341: No bootstrap required, opened a new log
Aug 08 13:15:28 dist-test-slave-49h3 krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:15:28.576870 22174 raft_consensus.cc:348] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260808 13:15:28.577412 22174 raft_consensus.cc:374] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:28.577562 22174 raft_consensus.cc:729] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c16e81583f4c4332a70d27c618fc3341, State: Initialized, Role: FOLLOWER
I20260808 13:15:28.578231 22174 consensus_queue.cc:260] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260808 13:15:28.578518 22174 raft_consensus.cc:388] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:28.578660 22174 raft_consensus.cc:482] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:28.578863 22174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:28.578877 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:28.556990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:51601 (local address 127.0.0.1:38713)
0808 13:15:28.557800 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:28.557811 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:28.557852 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:15:28.559549 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:28.559572 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:28.559951 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:28.560247 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:28.562067 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:28.563419 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:28.565076 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:28.565813 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:28.565942 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:28.570558 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:28.570569 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:28.570591 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:28.570646 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:28.574505 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:28.575291 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:28.575299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:28.575308 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:28.575504 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:28.575879 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:28.575884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:28.575888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:28.576561 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:28.576874 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:28.577188 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 13:15:28.577542 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":389,"threads_started":1}
I20260808 13:15:28.581298 22174 raft_consensus.cc:504] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260808 13:15:28.581840 22174 leader_election.cc:302] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c16e81583f4c4332a70d27c618fc3341; no voters:
I20260808 13:15:28.582404 22174 leader_election.cc:288] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:28.582558 22179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:28.582698 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:28.583155 22179 raft_consensus.cc:686] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [term 1 LEADER]: Becoming Leader. State: Replica: c16e81583f4c4332a70d27c618fc3341, State: Running, Role: LEADER
I20260808 13:15:28.583762 22179 consensus_queue.cc:237] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260808 13:15:28.585356 22179 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c16e81583f4c4332a70d27c618fc3341. Latest consensus state: current_term: 1 leader_uuid: "c16e81583f4c4332a70d27c618fc3341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } } }
I20260808 13:15:28.586867 22179 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:28.586537 22181 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c16e81583f4c4332a70d27c618fc3341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16e81583f4c4332a70d27c618fc3341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } } }
I20260808 13:15:28.587205 22181 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:28.592833 22174 sys_catalog.cc:564] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:28.593024 22184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:28.598449 22184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:28.609143 22184 catalog_manager.cc:1347] Generated new cluster ID: 89211e8a93b94eb0bfa1d9ff22fa3484
I20260808 13:15:28.609249 22184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:28.610781 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:28.591526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:59615 (local address 127.0.0.1:38713)
0808 13:15:28.591702 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:28.591707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:28.592277 (+ 570us) server_negotiation.cc:407] Connection header received
0808 13:15:28.592460 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:28.592465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:28.592551 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:28.592666 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:28.594602 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:28.595983 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:28.597990 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:28.598727 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:28.598846 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:28.602406 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:28.602413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:28.602419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:28.602461 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:28.606353 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:28.607349 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:28.607356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:28.607360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:28.607521 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:28.608075 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:28.608081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:28.608085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:28.608522 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:28.609995 (+ 1473us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:28.610209 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 13:15:28.610408 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:15:28.630764 22184 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:28.632543 22184 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:28.644269 22184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c16e81583f4c4332a70d27c618fc3341: Generated new TSK 0
I20260808 13:15:28.645268 22184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:28.659600 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22101
I20260808 13:15:28.660845 22168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:28.753582 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22101
W20260808 13:15:28.772431 22084 connection.cc:535] client connection to 127.0.0.1:38713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38713): Connection reset by peer (error 104)
2026-08-08T13:15:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:28 dist-test-slave-49h3 krb5kdc[22214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:28 dist-test-slave-49h3 krb5kdc[22214](info): set up 2 sockets
Aug 08 13:15:28 dist-test-slave-49h3 krb5kdc[22214](info): commencing operation
krb5kdc: starting...
W20260808 13:15:29.848062 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.006s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:30 dist-test-slave-49h3 krb5kdc[22214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:30.180469 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:30.519289 22230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:30.519608 22230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:30.519759 22230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:30.527084 22230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:30.527212 22230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:30.527295 22230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:30.527388 22230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:30.527447 22230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:30.537230 22230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:30.538584 22230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:30.539887 22230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:30.551350 22242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:30.551326 22239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:30.551456 22235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:30.552210 22230 server_base.cc:1034] running on GCE node
I20260808 13:15:30.553023 22230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:30.554522 22230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:30.555749 22230 hybrid_clock.cc:648] HybridClock initialized: now 1786194930555689 us; error 39 us; skew 500 ppm
I20260808 13:15:30.556133 22230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:30 dist-test-slave-49h3 krb5kdc[22214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194930, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:30.559329 22230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:30.560798 22230 webserver.cc:460] Webserver started at http://127.0.0.1:44751/ using document root <none> and password file <none>
I20260808 13:15:30.561515 22230 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:30.561645 22230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:30.561934 22230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:30.564898 22230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "c4c7d8960e82482c9b1a15854b1f2375"
format_stamp: "Formatted at 2026-08-08 13:15:30 on dist-test-slave-49h3"
server_key: "fe86795bf830887551985355dbbde818"
server_key_iv: "22ba3234e861e645c3b6812f1341565b"
server_key_version: "encryptionkey@0"
I20260808 13:15:30.565716 22230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "c4c7d8960e82482c9b1a15854b1f2375"
format_stamp: "Formatted at 2026-08-08 13:15:30 on dist-test-slave-49h3"
server_key: "fe86795bf830887551985355dbbde818"
server_key_iv: "22ba3234e861e645c3b6812f1341565b"
server_key_version: "encryptionkey@0"
I20260808 13:15:30.570969 22230 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:15:30.574398 22250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:30.575716 22230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:15:30.575896 22230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "c4c7d8960e82482c9b1a15854b1f2375"
format_stamp: "Formatted at 2026-08-08 13:15:30 on dist-test-slave-49h3"
server_key: "fe86795bf830887551985355dbbde818"
server_key_iv: "22ba3234e861e645c3b6812f1341565b"
server_key_version: "encryptionkey@0"
I20260808 13:15:30.576071 22230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:30.599159 22230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:30.603487 22230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:30.603778 22230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:30.621739 22230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39661
I20260808 13:15:30.621791 22305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39661 every 8 connection(s)
I20260808 13:15:30.623055 22230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:30.626646 22306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:30.632774 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22230
I20260808 13:15:30.632947 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:30.633368 31982 external_mini_cluster.cc:1428] Setting key d4ac5371d21aa25f7bb2797ff197c232
I20260808 13:15:30.637167 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375: Bootstrap starting.
I20260808 13:15:30.640551 22306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:30.641886 22306 log.cc:824] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:30.645277 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375: No bootstrap required, opened a new log
I20260808 13:15:30.647127 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:30.635266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:49141 (local address 127.0.0.1:39661)
0808 13:15:30.636017 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:30.636028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:30.636068 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:15:30.637831 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:30.637852 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:30.638221 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:30.638522 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:30.639177 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:30.640451 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:30.642335 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:30.643379 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:30.643501 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:30.644835 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:30.644843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:30.644856 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:30.645409 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:30.645497 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:30.645524 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:30.645645 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:30.645922 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 13:15:30.646107 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":336,"threads_started":1}
I20260808 13:15:30.650844 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:30.653270 22306 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } }
I20260808 13:15:30.653829 22306 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:30.653927 22306 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c7d8960e82482c9b1a15854b1f2375, State: Initialized, Role: FOLLOWER
I20260808 13:15:30.654556 22306 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } }
I20260808 13:15:30.654806 22306 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:30.654964 22306 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:30.655175 22306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:30.657712 22306 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } }
I20260808 13:15:30.658197 22306 leader_election.cc:302] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4c7d8960e82482c9b1a15854b1f2375; no voters:
I20260808 13:15:30.658823 22306 leader_election.cc:288] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:30.658943 22319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:30.661206 22306 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:30.661770 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:30.655111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:56919 (local address 127.0.0.1:39661)
0808 13:15:30.655280 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:30.655287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:30.655918 (+ 631us) server_negotiation.cc:407] Connection header received
0808 13:15:30.656115 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:30.656122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:30.656192 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:30.656285 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:30.656873 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:30.657673 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:30.659029 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:30.659699 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:30.659821 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:30.660931 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:30.660936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:30.660939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:30.660956 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:30.660994 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:30.661008 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:30.661098 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:30.661296 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 13:15:30.661353 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 13:15:30.665711 22319 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [term 1 LEADER]: Becoming Leader. State: Replica: c4c7d8960e82482c9b1a15854b1f2375, State: Running, Role: LEADER
I20260808 13:15:30.666417 22319 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } }
I20260808 13:15:30.669763 22322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4c7d8960e82482c9b1a15854b1f2375. Latest consensus state: current_term: 1 leader_uuid: "c4c7d8960e82482c9b1a15854b1f2375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } } }
I20260808 13:15:30.669726 22320 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4c7d8960e82482c9b1a15854b1f2375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c7d8960e82482c9b1a15854b1f2375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39661 } } }
I20260808 13:15:30.670197 22322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:30.670198 22320 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:30.671404 22330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:30.676893 22330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:30.686280 22330 catalog_manager.cc:1347] Generated new cluster ID: bff0f8aab02d43d4a29bd749c24c6979
I20260808 13:15:30.686385 22330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:30.722513 22330 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:30.723888 22330 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:30.731002 22330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4c7d8960e82482c9b1a15854b1f2375: Generated new TSK 0
I20260808 13:15:30.731840 22330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:30.740511 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22230
W20260808 13:15:30.742084 22301 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.148:49141
W20260808 13:15:30.742892 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22230: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:30.743026 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22230
W20260808 13:15:30.761543 22212 connection.cc:535] client connection to 127.0.0.1:39661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39661): Connection reset by peer (error 104)
2026-08-08T13:15:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1989 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:30 dist-test-slave-49h3 krb5kdc[22343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:30 dist-test-slave-49h3 krb5kdc[22343](info): set up 2 sockets
Aug 08 13:15:30 dist-test-slave-49h3 krb5kdc[22343](info): commencing operation
krb5kdc: starting...
W20260808 13:15:31.840871 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:32 dist-test-slave-49h3 krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:32.174919 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:32.515949 22356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:32.516266 22356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:32.516417 22356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:32.523779 22356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:32.523926 22356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:32.524006 22356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:32.524101 22356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:32.524168 22356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:32.534351 22356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:32.535610 22356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:32.536859 22356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:32.548249 22363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:32.548357 22365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:32.548717 22362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:32.549090 22356 server_base.cc:1034] running on GCE node
I20260808 13:15:32.549880 22356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:32.551208 22356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:32.552424 22356 hybrid_clock.cc:648] HybridClock initialized: now 1786194932552379 us; error 31 us; skew 500 ppm
I20260808 13:15:32.552759 22356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:32 dist-test-slave-49h3 krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194932, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:32.560292 22356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:32.561852 22356 webserver.cc:460] Webserver started at http://127.0.0.1:35825/ using document root <none> and password file <none>
I20260808 13:15:32.562525 22356 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:32.562647 22356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:32.562935 22356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:32.566095 22356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "f3e55026eaa14835990701dd4cee5959"
format_stamp: "Formatted at 2026-08-08 13:15:32 on dist-test-slave-49h3"
server_key: "cbff3a88c5b5916375cc53fae5fb5c0b"
server_key_iv: "4f4c64b8dbb859fa3f4d031e8db3a9d5"
server_key_version: "encryptionkey@0"
I20260808 13:15:32.566872 22356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "f3e55026eaa14835990701dd4cee5959"
format_stamp: "Formatted at 2026-08-08 13:15:32 on dist-test-slave-49h3"
server_key: "cbff3a88c5b5916375cc53fae5fb5c0b"
server_key_iv: "4f4c64b8dbb859fa3f4d031e8db3a9d5"
server_key_version: "encryptionkey@0"
I20260808 13:15:32.571935 22356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:32.575498 22372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:32.576838 22356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:32.577010 22356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "f3e55026eaa14835990701dd4cee5959"
format_stamp: "Formatted at 2026-08-08 13:15:32 on dist-test-slave-49h3"
server_key: "cbff3a88c5b5916375cc53fae5fb5c0b"
server_key_iv: "4f4c64b8dbb859fa3f4d031e8db3a9d5"
server_key_version: "encryptionkey@0"
I20260808 13:15:32.577203 22356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:32.582926 22356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:32.586983 22356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:32.587267 22356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:32.604454 22356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46497
I20260808 13:15:32.604491 22425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46497 every 8 connection(s)
I20260808 13:15:32.605859 22356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:32.609573 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:32.615811 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22356
I20260808 13:15:32.615980 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:32.616420 31982 external_mini_cluster.cc:1428] Setting key e1d510a2ef9fbb495fe679d0cfd17621
I20260808 13:15:32.619808 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959: Bootstrap starting.
I20260808 13:15:32.623258 22426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:32.623922 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:32.618116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:43561 (local address 127.0.0.1:46497)
0808 13:15:32.618898 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:32.618910 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:32.618950 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:15:32.620639 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:32.620660 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:32.620966 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:32.621283 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:32.622333 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:32.622339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:32.622351 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:32.622368 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:32.622421 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:32.622439 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:32.622575 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:32.622758 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 13:15:32.622934 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":329,"threads_started":1}
I20260808 13:15:32.624338 22426 log.cc:824] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:32.627381 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959: No bootstrap required, opened a new log
I20260808 13:15:32.627542 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:32.634647 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:32.632022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:56473 (local address 127.0.0.1:46497)
0808 13:15:32.632219 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:32.632233 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:32.632583 (+ 350us) server_negotiation.cc:407] Connection header received
0808 13:15:32.632720 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:32.632726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:32.632786 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:32.632888 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:32.633915 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:32.633921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:32.633926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:32.633945 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:32.633996 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:32.634018 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:32.634152 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:32.634293 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 13:15:32.634357 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:15:32.635478 22426 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } }
I20260808 13:15:32.635986 22426 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:32.636127 22426 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e55026eaa14835990701dd4cee5959, State: Initialized, Role: FOLLOWER
I20260808 13:15:32.636685 22426 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } }
I20260808 13:15:32.636929 22426 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:32.637054 22426 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:32.637210 22426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:32.639443 22426 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } }
I20260808 13:15:32.639921 22426 leader_election.cc:302] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3e55026eaa14835990701dd4cee5959; no voters:
I20260808 13:15:32.640489 22426 leader_election.cc:288] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:32.640635 22437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:32.641773 22437 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [term 1 LEADER]: Becoming Leader. State: Replica: f3e55026eaa14835990701dd4cee5959, State: Running, Role: LEADER
I20260808 13:15:32.642302 22437 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } }
I20260808 13:15:32.642704 22426 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:32.645071 22439 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e55026eaa14835990701dd4cee5959. Latest consensus state: current_term: 1 leader_uuid: "f3e55026eaa14835990701dd4cee5959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } } }
I20260808 13:15:32.646412 22439 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:32.645083 22438 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3e55026eaa14835990701dd4cee5959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e55026eaa14835990701dd4cee5959" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46497 } } }
I20260808 13:15:32.646597 22438 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:32.647475 22447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:32.653021 22447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:32.661710 22447 catalog_manager.cc:1347] Generated new cluster ID: d5ba2e146251430aa610b2cc895a4c3a
I20260808 13:15:32.661832 22447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:32.688877 22447 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:32.689371 22447 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:32.708077 22447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e55026eaa14835990701dd4cee5959: Generated new TSK 0
I20260808 13:15:32.709249 22447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:32.724615 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22356
W20260808 13:15:32.726143 22421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.148:43561
W20260808 13:15:32.726914 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22356: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:32.727062 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22356
2026-08-08T13:15:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1985 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:32 dist-test-slave-49h3 krb5kdc[22461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:32 dist-test-slave-49h3 krb5kdc[22461](info): set up 2 sockets
Aug 08 13:15:32 dist-test-slave-49h3 krb5kdc[22461](info): commencing operation
krb5kdc: starting...
W20260808 13:15:33.828914 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:34.168871 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:34.507022 22476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:34.507381 22476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:34.507534 22476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:34.514988 22476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:34.515129 22476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:34.515210 22476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:34.515298 22476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:34.515360 22476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:34.525550 22476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40697
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:34.526922 22476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:34.528255 22476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:34.539796 22483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:34.539742 22484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:34.539841 22486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:34.540617 22476 server_base.cc:1034] running on GCE node
I20260808 13:15:34.541390 22476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:34.542748 22476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:34.543941 22476 hybrid_clock.cc:648] HybridClock initialized: now 1786194934543898 us; error 34 us; skew 500 ppm
I20260808 13:15:34.544296 22476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194934, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:34.547421 22476 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:34.548892 22476 webserver.cc:460] Webserver started at http://127.0.0.1:44265/ using document root <none> and password file <none>
I20260808 13:15:34.549588 22476 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:34.549711 22476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:34.549994 22476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:34.553023 22476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "7b636974ab994f6ca5c4e7dc5cd9b977"
format_stamp: "Formatted at 2026-08-08 13:15:34 on dist-test-slave-49h3"
server_key: "b47b5f8d84e9dff93d48e262eb3e0a5a"
server_key_iv: "e9f4ab141b450649f4350473b701a3ee"
server_key_version: "encryptionkey@0"
I20260808 13:15:34.553823 22476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "7b636974ab994f6ca5c4e7dc5cd9b977"
format_stamp: "Formatted at 2026-08-08 13:15:34 on dist-test-slave-49h3"
server_key: "b47b5f8d84e9dff93d48e262eb3e0a5a"
server_key_iv: "e9f4ab141b450649f4350473b701a3ee"
server_key_version: "encryptionkey@0"
I20260808 13:15:34.558852 22476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:15:34.562263 22493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:34.563540 22476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:34.563702 22476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "7b636974ab994f6ca5c4e7dc5cd9b977"
format_stamp: "Formatted at 2026-08-08 13:15:34 on dist-test-slave-49h3"
server_key: "b47b5f8d84e9dff93d48e262eb3e0a5a"
server_key_iv: "e9f4ab141b450649f4350473b701a3ee"
server_key_version: "encryptionkey@0"
I20260808 13:15:34.563884 22476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:34.578318 22476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:34.582160 22476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:34.582430 22476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:34.600497 22476 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40697
I20260808 13:15:34.600538 22545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40697 every 8 connection(s)
I20260808 13:15:34.601917 22476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:34.605592 22546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:34.610749 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22476
I20260808 13:15:34.610925 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:34.611430 31982 external_mini_cluster.cc:1428] Setting key 9e5175a7aec3f5d31762c848c1142070
I20260808 13:15:34.616925 22546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977: Bootstrap starting.
I20260808 13:15:34.620615 22546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:34.621951 22546 log.cc:824] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:34.625414 22546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977: No bootstrap required, opened a new log
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:15:34.634685 22546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } }
I20260808 13:15:34.635095 22546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:34.635198 22546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b636974ab994f6ca5c4e7dc5cd9b977, State: Initialized, Role: FOLLOWER
I20260808 13:15:34.635464 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:34.613276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:55611 (local address 127.0.0.1:40697)
0808 13:15:34.614167 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:34.614182 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:34.614228 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:15:34.615971 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:34.615994 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:34.616374 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:34.616703 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:34.618539 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:34.619900 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:34.621736 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:34.622476 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:34.622686 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:34.627563 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:34.627570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:34.627588 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:34.627626 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:34.630782 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:34.631718 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:34.631725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:34.631732 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:34.631917 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:34.632480 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:34.632484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:34.632486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:34.633052 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:34.633334 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:34.633916 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 13:15:34.634285 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":415,"threads_started":1}
I20260808 13:15:34.635843 22546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } }
I20260808 13:15:34.636087 22546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:34.636220 22546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:34.636399 22546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:34.638964 22546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } }
I20260808 13:15:34.639335 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:34.639504 22546 leader_election.cc:302] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b636974ab994f6ca5c4e7dc5cd9b977; no voters:
I20260808 13:15:34.640117 22546 leader_election.cc:288] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:34.640269 22551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:34.641705 22551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [term 1 LEADER]: Becoming Leader. State: Replica: 7b636974ab994f6ca5c4e7dc5cd9b977, State: Running, Role: LEADER
I20260808 13:15:34.642336 22546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:34.642267 22551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } }
I20260808 13:15:34.650818 22552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } } }
I20260808 13:15:34.651222 22552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:34.651688 22553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b636974ab994f6ca5c4e7dc5cd9b977. Latest consensus state: current_term: 1 leader_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b636974ab994f6ca5c4e7dc5cd9b977" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40697 } } }
I20260808 13:15:34.652092 22553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:34.658600 22572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:34.664036 22572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:34.666759 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:34.645223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:44953 (local address 127.0.0.1:40697)
0808 13:15:34.645501 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:34.645508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:34.645532 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:15:34.645591 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:34.645596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:34.645677 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:34.645784 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:34.647816 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:34.649104 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:34.651094 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:34.652013 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:34.652152 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:34.660636 (+ 8484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:34.660642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:34.660645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:34.660681 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:34.663597 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:34.664474 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:34.664479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:34.664481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:34.664605 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:34.665068 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:34.665071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:34.665073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:34.665530 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:34.665810 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:34.666221 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 13:15:34.666416 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260808 13:15:34.674068 22572 catalog_manager.cc:1347] Generated new cluster ID: 29b88d109e7249fcb824bfbe085ffc89
I20260808 13:15:34.674258 22572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:34.698661 22572 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:34.700547 22572 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:34.711598 22572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b636974ab994f6ca5c4e7dc5cd9b977: Generated new TSK 0
I20260808 13:15:34.712435 22572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:34.726959 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22476
I20260808 13:15:34.728228 22541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:34.821990 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22476
W20260808 13:15:34.840261 22459 connection.cc:535] client connection to 127.0.0.1:40697 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40697): Connection reset by peer (error 104)
2026-08-08T13:15:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2093 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22581](info): set up 2 sockets
Aug 08 13:15:34 dist-test-slave-49h3 krb5kdc[22581](info): commencing operation
krb5kdc: starting...
W20260808 13:15:35.913281 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:36 dist-test-slave-49h3 krb5kdc[22581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:36.254681 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:36.597096 22598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:36.597404 22598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:36.597546 22598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:36.604802 22598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:36.604919 22598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:36.604970 22598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:36.605044 22598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:36.605089 22598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:36.615212 22598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:36.616495 22598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:36.617794 22598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:36.629097 22606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:36.629119 22604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:36.629369 22603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:36.629752 22598 server_base.cc:1034] running on GCE node
I20260808 13:15:36.630616 22598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:36.631994 22598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:36.633219 22598 hybrid_clock.cc:648] HybridClock initialized: now 1786194936633158 us; error 47 us; skew 500 ppm
I20260808 13:15:36.633598 22598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:36 dist-test-slave-49h3 krb5kdc[22581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194936, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:36.636669 22598 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:36.638208 22598 webserver.cc:460] Webserver started at http://127.0.0.1:45701/ using document root <none> and password file <none>
I20260808 13:15:36.638866 22598 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:36.638989 22598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:36.639297 22598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:36.642447 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "f4b20538e7df4b87a7dc351cc6102397"
format_stamp: "Formatted at 2026-08-08 13:15:36 on dist-test-slave-49h3"
server_key: "5c06d65b2fc8a03750ba84247c199ee2"
server_key_iv: "38dceb311aba1c6f53cd74fce4c13ad3"
server_key_version: "encryptionkey@0"
I20260808 13:15:36.643209 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "f4b20538e7df4b87a7dc351cc6102397"
format_stamp: "Formatted at 2026-08-08 13:15:36 on dist-test-slave-49h3"
server_key: "5c06d65b2fc8a03750ba84247c199ee2"
server_key_iv: "38dceb311aba1c6f53cd74fce4c13ad3"
server_key_version: "encryptionkey@0"
I20260808 13:15:36.648476 22598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:15:36.651919 22614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:36.653230 22598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:36.653393 22598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "f4b20538e7df4b87a7dc351cc6102397"
format_stamp: "Formatted at 2026-08-08 13:15:36 on dist-test-slave-49h3"
server_key: "5c06d65b2fc8a03750ba84247c199ee2"
server_key_iv: "38dceb311aba1c6f53cd74fce4c13ad3"
server_key_version: "encryptionkey@0"
I20260808 13:15:36.653620 22598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:36.670782 22598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:36.674980 22598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:36.675271 22598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:36.693512 22598 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44701
I20260808 13:15:36.693536 22671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44701 every 8 connection(s)
I20260808 13:15:36.694893 22598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:36.698030 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22598
I20260808 13:15:36.698176 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:36.698685 31982 external_mini_cluster.cc:1428] Setting key 762cfc7105e28a1d7a90ae0e5633b4c8
I20260808 13:15:36.698881 22672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:36.714423 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:36.700596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:57061 (local address 127.0.0.1:44701)
0808 13:15:36.701409 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:36.701428 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:36.701507 (+ 79us) server_negotiation.cc:407] Connection header received
0808 13:15:36.703762 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:36.703793 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:36.704369 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:36.704870 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:36.705427 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:36.707504 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:36.709030 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:36.710132 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:36.710338 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:36.711578 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:36.711592 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:36.711612 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:36.712377 (+ 765us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:36.712478 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:36.712509 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:36.712650 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:36.712874 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:15:36.713060 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":327,"threads_started":1}
I20260808 13:15:36.714435 22672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397: Bootstrap starting.
I20260808 13:15:36.717868 22672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:36.718626 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:36.719030 22672 log.cc:824] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:36.722317 22672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397: No bootstrap required, opened a new log
I20260808 13:15:36.730440 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:36.723715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:43695 (local address 127.0.0.1:44701)
0808 13:15:36.723887 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:36.723893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:36.724255 (+ 362us) server_negotiation.cc:407] Connection header received
0808 13:15:36.724413 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:36.724419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:36.724495 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:36.724615 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:36.725125 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:36.726406 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:36.727727 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:36.728445 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:36.728577 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:36.729629 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:36.729636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:36.729643 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:36.729666 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:36.729733 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:36.729758 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:36.729893 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:36.730041 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 13:15:36.730098 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:15:36.731983 22672 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260808 13:15:36.732360 22672 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:36.732460 22672 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b20538e7df4b87a7dc351cc6102397, State: Initialized, Role: FOLLOWER
I20260808 13:15:36.733017 22672 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260808 13:15:36.733249 22672 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:36.733382 22672 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:36.733566 22672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:36.735826 22672 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260808 13:15:36.736320 22672 leader_election.cc:302] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4b20538e7df4b87a7dc351cc6102397; no voters:
I20260808 13:15:36.736879 22672 leader_election.cc:288] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:36.737010 22682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:36.737881 22682 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [term 1 LEADER]: Becoming Leader. State: Replica: f4b20538e7df4b87a7dc351cc6102397, State: Running, Role: LEADER
I20260808 13:15:36.738584 22682 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260808 13:15:36.739089 22672 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:36.741572 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b20538e7df4b87a7dc351cc6102397. Latest consensus state: current_term: 1 leader_uuid: "f4b20538e7df4b87a7dc351cc6102397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260808 13:15:36.741972 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:36.742395 22683 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4b20538e7df4b87a7dc351cc6102397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b20538e7df4b87a7dc351cc6102397" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260808 13:15:36.742633 22683 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:36.746987 22692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:36.752122 22692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:36.761579 22692 catalog_manager.cc:1347] Generated new cluster ID: 714ae041852f4f2b914b60eeb73c6311
I20260808 13:15:36.761718 22692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:36.786732 22692 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:36.788058 22692 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:36.794528 22692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4b20538e7df4b87a7dc351cc6102397: Generated new TSK 0
I20260808 13:15:36.795392 22692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:36.800827 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22598
W20260808 13:15:36.802258 22665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.148:57061
W20260808 13:15:36.802989 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:36.803150 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22598
W20260808 13:15:36.821946 22579 connection.cc:535] client connection to 127.0.0.1:44701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44701): Connection reset by peer (error 104)
2026-08-08T13:15:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1981 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:36 dist-test-slave-49h3 krb5kdc[22705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:36 dist-test-slave-49h3 krb5kdc[22705](info): set up 2 sockets
Aug 08 13:15:36 dist-test-slave-49h3 krb5kdc[22705](info): commencing operation
krb5kdc: starting...
W20260808 13:15:37.904950 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:38 dist-test-slave-49h3 krb5kdc[22705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:15:38.242323 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:38.580408 22724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:38.580691 22724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:38.580826 22724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:38.587966 22724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:38.588095 22724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:38.588176 22724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:38.588270 22724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:38.588321 22724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:38.598110 22724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36117
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:38.599386 22724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:38.600647 22724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:38.611727 22729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:38.611804 22732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:38.611765 22730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:38.612653 22724 server_base.cc:1034] running on GCE node
I20260808 13:15:38.613396 22724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:38.614794 22724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:38.616000 22724 hybrid_clock.cc:648] HybridClock initialized: now 1786194938615935 us; error 52 us; skew 500 ppm
I20260808 13:15:38.616329 22724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:38 dist-test-slave-49h3 krb5kdc[22705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194938, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:38.622325 22724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:15:38.623852 22724 webserver.cc:460] Webserver started at http://127.0.0.1:44357/ using document root <none> and password file <none>
I20260808 13:15:38.624511 22724 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:38.624629 22724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:38.624918 22724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:38.627954 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "67892118dcce41aeb58334e6962cc642"
format_stamp: "Formatted at 2026-08-08 13:15:38 on dist-test-slave-49h3"
server_key: "85bdbbb5a9ce77e0b000c295a67806da"
server_key_iv: "83d1b4dce6f1dc46dd4ffcbe5d78068b"
server_key_version: "encryptionkey@0"
I20260808 13:15:38.628715 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "67892118dcce41aeb58334e6962cc642"
format_stamp: "Formatted at 2026-08-08 13:15:38 on dist-test-slave-49h3"
server_key: "85bdbbb5a9ce77e0b000c295a67806da"
server_key_iv: "83d1b4dce6f1dc46dd4ffcbe5d78068b"
server_key_version: "encryptionkey@0"
I20260808 13:15:38.633971 22724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:15:38.637388 22740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:38.638622 22724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:38.638787 22724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "67892118dcce41aeb58334e6962cc642"
format_stamp: "Formatted at 2026-08-08 13:15:38 on dist-test-slave-49h3"
server_key: "85bdbbb5a9ce77e0b000c295a67806da"
server_key_iv: "83d1b4dce6f1dc46dd4ffcbe5d78068b"
server_key_version: "encryptionkey@0"
I20260808 13:15:38.638974 22724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:38.644634 22724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:38.648617 22724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:38.648904 22724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:38.666741 22724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36117
I20260808 13:15:38.666793 22795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36117 every 8 connection(s)
I20260808 13:15:38.668146 22724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:38.671881 22797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:38.674573 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22724
I20260808 13:15:38.674784 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:38.675383 31982 external_mini_cluster.cc:1428] Setting key af97919f83e45dca9a2ae8bf8c522cf0
I20260808 13:15:38.683085 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:38.677228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:59099 (local address 127.0.0.1:36117)
0808 13:15:38.678036 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:38.678047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:38.678087 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:15:38.679755 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:38.679782 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:38.680114 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:38.680449 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:38.681562 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:38.681571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:38.681586 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:38.681602 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:38.681664 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:38.681688 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:38.681868 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:38.681971 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 13:15:38.682157 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":359,"threads_started":1}
I20260808 13:15:38.683925 22797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642: Bootstrap starting.
I20260808 13:15:38.686595 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:38.687199 22797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:38.688410 22797 log.cc:824] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:38.691612 22797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642: No bootstrap required, opened a new log
I20260808 13:15:38.693949 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:38.691551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:46201 (local address 127.0.0.1:36117)
0808 13:15:38.691735 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:38.691741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:38.692114 (+ 373us) server_negotiation.cc:407] Connection header received
0808 13:15:38.692287 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:38.692290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:38.692338 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:38.692413 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:38.693303 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:38.693307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:38.693310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:38.693323 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:38.693360 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:38.693377 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:38.693498 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:38.693629 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 13:15:38.693680 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:15:38.699462 22797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } }
I20260808 13:15:38.699806 22797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:38.699894 22797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67892118dcce41aeb58334e6962cc642, State: Initialized, Role: FOLLOWER
I20260808 13:15:38.700423 22797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } }
I20260808 13:15:38.700657 22797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:38.700798 22797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:38.700968 22797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:38.703621 22797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } }
I20260808 13:15:38.704087 22797 leader_election.cc:302] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67892118dcce41aeb58334e6962cc642; no voters:
I20260808 13:15:38.704672 22797 leader_election.cc:288] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:38.704790 22809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:38.705403 22809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [term 1 LEADER]: Becoming Leader. State: Replica: 67892118dcce41aeb58334e6962cc642, State: Running, Role: LEADER
I20260808 13:15:38.705981 22809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } }
I20260808 13:15:38.706753 22797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:38.708667 22811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67892118dcce41aeb58334e6962cc642. Latest consensus state: current_term: 1 leader_uuid: "67892118dcce41aeb58334e6962cc642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } } }
I20260808 13:15:38.709060 22811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:38.710623 22818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:38.710215 22810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67892118dcce41aeb58334e6962cc642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67892118dcce41aeb58334e6962cc642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36117 } } }
I20260808 13:15:38.711638 22810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:38.718659 22818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:38.726895 22818 catalog_manager.cc:1347] Generated new cluster ID: 62b3417a8a5040ef849dc53b1b715a75
I20260808 13:15:38.727011 22818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:38.759641 22818 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:38.760084 22818 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:38.771260 22818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67892118dcce41aeb58334e6962cc642: Generated new TSK 0
I20260808 13:15:38.772089 22818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:38.790561 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22724
W20260808 13:15:38.791635 22791 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.148:59099
W20260808 13:15:38.792382 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22724: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:38.792531 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22724
2026-08-08T13:15:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1990 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:38 dist-test-slave-49h3 krb5kdc[22833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:38 dist-test-slave-49h3 krb5kdc[22833](info): set up 2 sockets
Aug 08 13:15:38 dist-test-slave-49h3 krb5kdc[22833](info): commencing operation
krb5kdc: starting...
W20260808 13:15:39.892921 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:40.224630 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39307
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:45825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:40.563222 22848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:40.563558 22848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:40.563678 22848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:40.571161 22848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:40.571283 22848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:40.571338 22848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:40.571403 22848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:40.571446 22848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:40.581491 22848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:45825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:39307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:40.582820 22848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:40.584174 22848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:40.595919 22853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:40.595857 22856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:40.595825 22854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:40.596949 22848 server_base.cc:1034] running on GCE node
I20260808 13:15:40.597754 22848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:40.599124 22848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:40.600320 22848 hybrid_clock.cc:648] HybridClock initialized: now 1786194940600278 us; error 32 us; skew 500 ppm
I20260808 13:15:40.600663 22848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:40.603583 22848 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:40.605083 22848 webserver.cc:460] Webserver started at http://127.31.59.190:44511/ using document root <none> and password file <none>
I20260808 13:15:40.605801 22848 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:40.605922 22848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:40.606225 22848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:40.609171 22848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "d3b2384035004b9dbe2bc70cc1499dfe"
format_stamp: "Formatted at 2026-08-08 13:15:40 on dist-test-slave-49h3"
server_key: "e151ef68d268a90cb542cc336678a999"
server_key_iv: "fd6c9a17f86819caf6c23ad3e2ce5640"
server_key_version: "encryptionkey@0"
I20260808 13:15:40.609933 22848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "d3b2384035004b9dbe2bc70cc1499dfe"
format_stamp: "Formatted at 2026-08-08 13:15:40 on dist-test-slave-49h3"
server_key: "e151ef68d268a90cb542cc336678a999"
server_key_iv: "fd6c9a17f86819caf6c23ad3e2ce5640"
server_key_version: "encryptionkey@0"
I20260808 13:15:40.614847 22848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 13:15:40.618196 22865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:40.619501 22848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:40.619678 22848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "d3b2384035004b9dbe2bc70cc1499dfe"
format_stamp: "Formatted at 2026-08-08 13:15:40 on dist-test-slave-49h3"
server_key: "e151ef68d268a90cb542cc336678a999"
server_key_iv: "fd6c9a17f86819caf6c23ad3e2ce5640"
server_key_version: "encryptionkey@0"
I20260808 13:15:40.619870 22848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:40.639690 22848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:40.643674 22848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:40.643939 22848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:40.662031 22848 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:39307
I20260808 13:15:40.662065 22923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:39307 every 8 connection(s)
I20260808 13:15:40.663363 22848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:40.666268 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22848
I20260808 13:15:40.666411 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:40.666852 31982 external_mini_cluster.cc:1428] Setting key cb7bc542f84283269f68e6194c5283b3
I20260808 13:15:40.667245 22924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:40.681067 22924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe: Bootstrap starting.
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:40.684824 22924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:40.686239 22924 log.cc:824] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:40.689675 22924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe: No bootstrap required, opened a new log
I20260808 13:15:40.691581 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:40.668588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60923 (local address 127.31.59.190:39307)
0808 13:15:40.669408 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:40.669423 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:40.669494 (+ 71us) server_negotiation.cc:407] Connection header received
0808 13:15:40.671923 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:40.671951 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:40.672461 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:40.672905 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:40.674270 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:40.676236 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:40.677505 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:40.678567 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:40.678772 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:40.682786 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:40.682798 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:40.682817 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:40.682868 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:40.686775 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:40.687830 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:40.687838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:40.687846 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:40.688062 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:40.688478 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:40.688483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:40.688486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:40.689194 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:40.689533 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:40.689897 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 13:15:40.690239 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":334,"threads_started":1}
I20260808 13:15:40.695303 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:40.698001 22924 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } }
I20260808 13:15:40.698364 22924 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:40.698475 22924 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b2384035004b9dbe2bc70cc1499dfe, State: Initialized, Role: FOLLOWER
I20260808 13:15:40.699014 22924 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } }
I20260808 13:15:40.699234 22924 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:40.699386 22924 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:40.699590 22924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:40.702847 22924 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } }
I20260808 13:15:40.703395 22924 leader_election.cc:302] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3b2384035004b9dbe2bc70cc1499dfe; no voters:
I20260808 13:15:40.704053 22924 leader_election.cc:288] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:40.704139 22936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:40.705730 22936 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [term 1 LEADER]: Becoming Leader. State: Replica: d3b2384035004b9dbe2bc70cc1499dfe, State: Running, Role: LEADER
I20260808 13:15:40.706305 22936 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } }
I20260808 13:15:40.706555 22924 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:40.709772 22938 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3b2384035004b9dbe2bc70cc1499dfe. Latest consensus state: current_term: 1 leader_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } } }
I20260808 13:15:40.710199 22938 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:40.710783 22937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2384035004b9dbe2bc70cc1499dfe" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39307 } } }
I20260808 13:15:40.711162 22937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:40.717551 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:40.699603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56151 (local address 127.31.59.190:39307)
0808 13:15:40.699797 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:40.699804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:40.700249 (+ 445us) server_negotiation.cc:407] Connection header received
0808 13:15:40.700425 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:40.700428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:40.700490 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:40.700576 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:40.702387 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:40.703169 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:40.704951 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:40.705651 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:40.705786 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:40.709472 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:40.709480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:40.709486 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:40.709530 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:40.713515 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:40.714590 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:40.714597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:40.714602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:40.714775 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:40.715376 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:40.715382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:40.715386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:40.715882 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:40.716167 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:40.716928 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 13:15:40.717136 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:15:40.719651 22941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:40.725265 22941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:40.734171 22941 catalog_manager.cc:1347] Generated new cluster ID: 350109834b7347f6b22483ad3da3b1a3
I20260808 13:15:40.734305 22941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:40.746155 22941 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:40.747577 22941 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:40.756913 22941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3b2384035004b9dbe2bc70cc1499dfe: Generated new TSK 0
I20260808 13:15:40.757786 22941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:40.769832 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22848
I20260808 13:15:40.770995 22919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:40.863490 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22848
W20260808 13:15:40.881907 22830 connection.cc:535] client connection to 127.31.59.190:39307 recv error: Network error: failed to read from TLS socket (remote: 127.31.59.190:39307): Connection reset by peer (error 104)
2026-08-08T13:15:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2068 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22960](info): set up 2 sockets
Aug 08 13:15:40 dist-test-slave-49h3 krb5kdc[22960](info): commencing operation
krb5kdc: starting...
W20260808 13:15:41.963452 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:42 dist-test-slave-49h3 krb5kdc[22960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:42.293885 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40461
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:40461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:42.635056 22974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:42.635407 22974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:42.635532 22974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:42.642706 22974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:42.642812 22974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:42.642864 22974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:42.642925 22974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:42.642966 22974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:42.652860 22974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:40461
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:40461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:42.654160 22974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:42.655484 22974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:42.666901 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:42.666942 22980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:42.666975 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:42.667764 22974 server_base.cc:1034] running on GCE node
I20260808 13:15:42.668555 22974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:42.670013 22974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:42.671217 22974 hybrid_clock.cc:648] HybridClock initialized: now 1786194942671168 us; error 42 us; skew 500 ppm
I20260808 13:15:42.671553 22974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:42 dist-test-slave-49h3 krb5kdc[22960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:42.674645 22974 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:42.676087 22974 webserver.cc:460] Webserver started at http://127.31.59.190:44865/ using document root <none> and password file <none>
I20260808 13:15:42.676739 22974 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:42.676859 22974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:42.677143 22974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:42.680110 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6"
format_stamp: "Formatted at 2026-08-08 13:15:42 on dist-test-slave-49h3"
server_key: "625060ce1f764cd3ce8ff52904adb986"
server_key_iv: "eb0e87ae082e073dc148299485b31605"
server_key_version: "encryptionkey@0"
I20260808 13:15:42.680846 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6"
format_stamp: "Formatted at 2026-08-08 13:15:42 on dist-test-slave-49h3"
server_key: "625060ce1f764cd3ce8ff52904adb986"
server_key_iv: "eb0e87ae082e073dc148299485b31605"
server_key_version: "encryptionkey@0"
I20260808 13:15:42.685900 22974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:15:42.689296 22992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:42.690587 22974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:42.690759 22974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6"
format_stamp: "Formatted at 2026-08-08 13:15:42 on dist-test-slave-49h3"
server_key: "625060ce1f764cd3ce8ff52904adb986"
server_key_iv: "eb0e87ae082e073dc148299485b31605"
server_key_version: "encryptionkey@0"
I20260808 13:15:42.690939 22974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:42.708055 22974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:42.712126 22974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:42.712405 22974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:42.730168 22974 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:40461
I20260808 13:15:42.730192 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:40461 every 8 connection(s)
I20260808 13:15:42.731575 22974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:42.735231 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:42.737330 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 22974
I20260808 13:15:42.737519 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:42.737941 31982 external_mini_cluster.cc:1428] Setting key 487a4ae4355c66f9e4a5df032e8793ac
I20260808 13:15:42.748371 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6: Bootstrap starting.
I20260808 13:15:42.752295 23049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:42.753015 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:42.739784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50325 (local address 127.31.59.190:40461)
0808 13:15:42.740654 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:42.740670 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:42.740725 (+ 55us) server_negotiation.cc:407] Connection header received
0808 13:15:42.743090 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:42.743118 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:42.743613 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:42.744034 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:42.744548 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:42.746621 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:42.748011 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:42.749012 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:42.749187 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:42.750308 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:42.750320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:42.750338 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:42.751044 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:42.751151 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:42.751187 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:42.751349 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:42.751515 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 13:15:42.751690 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":342,"threads_started":1}
I20260808 13:15:42.753656 23049 log.cc:824] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:42.756780 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6: No bootstrap required, opened a new log
I20260808 13:15:42.756975 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:42.766646 23049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } }
I20260808 13:15:42.767179 23049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:42.767295 23049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f3a6f52f5b4b6396f5ae0fa84689b6, State: Initialized, Role: FOLLOWER
I20260808 13:15:42.767956 23049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } }
I20260808 13:15:42.768297 23049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:42.768507 23049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:42.768713 23049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:42.768908 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:42.762264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59823 (local address 127.31.59.190:40461)
0808 13:15:42.762451 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:42.762460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:42.762764 (+ 304us) server_negotiation.cc:407] Connection header received
0808 13:15:42.762911 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:42.762920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:42.763004 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:42.763111 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:42.763650 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:42.764817 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:42.766134 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:42.766769 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:42.766886 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:42.768019 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:42.768026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:42.768030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:42.768053 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:42.768100 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:42.768120 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:42.768233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:42.768506 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 13:15:42.768562 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:15:42.771642 23049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } }
I20260808 13:15:42.772217 23049 leader_election.cc:302] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69f3a6f52f5b4b6396f5ae0fa84689b6; no voters:
I20260808 13:15:42.772891 23049 leader_election.cc:288] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:42.772964 23060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:42.773638 23060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [term 1 LEADER]: Becoming Leader. State: Replica: 69f3a6f52f5b4b6396f5ae0fa84689b6, State: Running, Role: LEADER
I20260808 13:15:42.774257 23060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } }
I20260808 13:15:42.775003 23049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:42.776801 23062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f3a6f52f5b4b6396f5ae0fa84689b6. Latest consensus state: current_term: 1 leader_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } } }
I20260808 13:15:42.776773 23061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f3a6f52f5b4b6396f5ae0fa84689b6" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 40461 } } }
I20260808 13:15:42.777215 23061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:42.777812 23062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:42.779316 23065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:42.783990 23065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:42.792177 23065 catalog_manager.cc:1347] Generated new cluster ID: 0c125647154a4ee491110bb9f3e96a29
I20260808 13:15:42.792300 23065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:42.805596 23065 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:42.807241 23065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:42.813953 23065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f3a6f52f5b4b6396f5ae0fa84689b6: Generated new TSK 0
I20260808 13:15:42.814775 23065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:42.824702 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22974
W20260808 13:15:42.826169 23044 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50325
W20260808 13:15:42.826898 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 22974: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:42.827062 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 22974
W20260808 13:15:42.846649 22958 connection.cc:535] client connection to 127.31.59.190:40461 recv error: Network error: failed to read from TLS socket (remote: 127.31.59.190:40461): Connection reset by peer (error 104)
2026-08-08T13:15:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1964 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:42 dist-test-slave-49h3 krb5kdc[23082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:42 dist-test-slave-49h3 krb5kdc[23082](info): set up 2 sockets
Aug 08 13:15:42 dist-test-slave-49h3 krb5kdc[23082](info): commencing operation
krb5kdc: starting...
W20260808 13:15:43.941643 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:44 dist-test-slave-49h3 krb5kdc[23082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:44.280074 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:35549
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:35549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:44.616721 23096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:44.617040 23096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:44.617180 23096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:44.624548 23096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:44.624672 23096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:44.624763 23096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:44.624847 23096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:44.624907 23096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:44.634925 23096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:35549
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:35549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:44.636260 23096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:44.637614 23096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:44.649075 23101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:44.649142 23102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:44.648996 23104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:44.650023 23096 server_base.cc:1034] running on GCE node
I20260808 13:15:44.650760 23096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:44.652140 23096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:44.653345 23096 hybrid_clock.cc:648] HybridClock initialized: now 1786194944653293 us; error 45 us; skew 500 ppm
I20260808 13:15:44.653700 23096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:44 dist-test-slave-49h3 krb5kdc[23082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194944, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:44.660017 23096 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:44.661624 23096 webserver.cc:460] Webserver started at http://127.31.59.190:46273/ using document root <none> and password file <none>
I20260808 13:15:44.662323 23096 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:44.662444 23096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:44.662756 23096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:44.665839 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "2ce2a39304234a96b4bf3630d5f39c5e"
format_stamp: "Formatted at 2026-08-08 13:15:44 on dist-test-slave-49h3"
server_key: "b698171b399985315307a1532e55ce80"
server_key_iv: "32330bdeb64a7a3e0cefaad9f6f48eb6"
server_key_version: "encryptionkey@0"
I20260808 13:15:44.666589 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "2ce2a39304234a96b4bf3630d5f39c5e"
format_stamp: "Formatted at 2026-08-08 13:15:44 on dist-test-slave-49h3"
server_key: "b698171b399985315307a1532e55ce80"
server_key_iv: "32330bdeb64a7a3e0cefaad9f6f48eb6"
server_key_version: "encryptionkey@0"
I20260808 13:15:44.671759 23096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:44.675206 23111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:44.676486 23096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:15:44.676656 23096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2ce2a39304234a96b4bf3630d5f39c5e"
format_stamp: "Formatted at 2026-08-08 13:15:44 on dist-test-slave-49h3"
server_key: "b698171b399985315307a1532e55ce80"
server_key_iv: "32330bdeb64a7a3e0cefaad9f6f48eb6"
server_key_version: "encryptionkey@0"
I20260808 13:15:44.676843 23096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:44.682648 23096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:44.686597 23096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:44.686890 23096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:44.704763 23096 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:35549
I20260808 13:15:44.704795 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:35549 every 8 connection(s)
I20260808 13:15:44.706256 23096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:44.710072 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:44.712121 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23096
I20260808 13:15:44.712329 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:44.712888 31982 external_mini_cluster.cc:1428] Setting key 9cb23d3113b3af1b792d8b79047fe4aa
I20260808 13:15:44.720438 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:44.714697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34599 (local address 127.31.59.190:35549)
0808 13:15:44.715547 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:44.715558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:44.715599 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:15:44.717204 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:44.717228 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:44.717619 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:44.717933 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:44.718977 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:44.718983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:44.718995 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:44.719008 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:44.719061 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:44.719079 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:44.719257 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:44.719346 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 13:15:44.719499 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":409,"threads_started":1}
I20260808 13:15:44.723640 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e: Bootstrap starting.
I20260808 13:15:44.724030 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:44.727504 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:44.728870 23170 log.cc:824] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:44.731887 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:44.729413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46941 (local address 127.31.59.190:35549)
0808 13:15:44.729632 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:44.729638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:44.730112 (+ 474us) server_negotiation.cc:407] Connection header received
0808 13:15:44.730296 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:44.730301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:44.730353 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:44.730457 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:44.731299 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:44.731304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:44.731309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:44.731327 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:44.731375 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:44.731394 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:44.731507 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:44.731554 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 13:15:44.731611 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 13:15:44.732139 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e: No bootstrap required, opened a new log
I20260808 13:15:44.739696 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } }
I20260808 13:15:44.740067 23170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:44.740168 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce2a39304234a96b4bf3630d5f39c5e, State: Initialized, Role: FOLLOWER
I20260808 13:15:44.740739 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } }
I20260808 13:15:44.740971 23170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:44.741104 23170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:44.741271 23170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:44.743611 23170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } }
I20260808 13:15:44.744122 23170 leader_election.cc:302] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ce2a39304234a96b4bf3630d5f39c5e; no voters:
I20260808 13:15:44.744702 23170 leader_election.cc:288] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:44.744834 23182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:44.745467 23182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [term 1 LEADER]: Becoming Leader. State: Replica: 2ce2a39304234a96b4bf3630d5f39c5e, State: Running, Role: LEADER
I20260808 13:15:44.746003 23182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } }
I20260808 13:15:44.746704 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:44.748675 23184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ce2a39304234a96b4bf3630d5f39c5e. Latest consensus state: current_term: 1 leader_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } } }
I20260808 13:15:44.749075 23184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:44.749047 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce2a39304234a96b4bf3630d5f39c5e" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 35549 } } }
I20260808 13:15:44.749346 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:44.751379 23188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:44.756330 23188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:44.764951 23188 catalog_manager.cc:1347] Generated new cluster ID: f222470b0f5240e8854e97bb3c1ba099
I20260808 13:15:44.765071 23188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:44.784197 23188 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:44.784706 23188 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:44.797034 23188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ce2a39304234a96b4bf3630d5f39c5e: Generated new TSK 0
I20260808 13:15:44.798168 23188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:44.809525 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23096
W20260808 13:15:44.811053 23163 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34599
W20260808 13:15:44.811764 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23096: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:44.811904 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23096
2026-08-08T13:15:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1983 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:44 dist-test-slave-49h3 krb5kdc[23204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:44 dist-test-slave-49h3 krb5kdc[23204](info): set up 2 sockets
Aug 08 13:15:44 dist-test-slave-49h3 krb5kdc[23204](info): commencing operation
krb5kdc: starting...
W20260808 13:15:45.911756 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:46 dist-test-slave-49h3 krb5kdc[23204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:46.257139 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42935
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:36173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:42935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:46.597574 23219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:46.597856 23219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:46.597992 23219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:46.605207 23219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:46.605327 23219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:46.605409 23219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:46.605520 23219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:46.605581 23219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:46.615352 23219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:36173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:42935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:42935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:46.616643 23219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:46.618011 23219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:46.629510 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:46.629557 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:46.629580 23227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:46.630656 23219 server_base.cc:1034] running on GCE node
I20260808 13:15:46.631491 23219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:46.632958 23219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:46.634159 23219 hybrid_clock.cc:648] HybridClock initialized: now 1786194946634114 us; error 39 us; skew 500 ppm
I20260808 13:15:46.634518 23219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:46 dist-test-slave-49h3 krb5kdc[23204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194946, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:46.637677 23219 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:46.639256 23219 webserver.cc:460] Webserver started at http://127.31.59.190:33345/ using document root <none> and password file <none>
I20260808 13:15:46.639920 23219 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:46.640028 23219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:46.640300 23219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:46.643388 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "ec80c53ab3114c909bd43fb20afa50a3"
format_stamp: "Formatted at 2026-08-08 13:15:46 on dist-test-slave-49h3"
server_key: "6a80a22e7beefa16070f7df676690e86"
server_key_iv: "f6f0a5d8093fb89b0fb94d8f3d8eb843"
server_key_version: "encryptionkey@0"
I20260808 13:15:46.644111 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "ec80c53ab3114c909bd43fb20afa50a3"
format_stamp: "Formatted at 2026-08-08 13:15:46 on dist-test-slave-49h3"
server_key: "6a80a22e7beefa16070f7df676690e86"
server_key_iv: "f6f0a5d8093fb89b0fb94d8f3d8eb843"
server_key_version: "encryptionkey@0"
I20260808 13:15:46.649227 23219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:15:46.652709 23235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:46.654070 23219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:15:46.654228 23219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "ec80c53ab3114c909bd43fb20afa50a3"
format_stamp: "Formatted at 2026-08-08 13:15:46 on dist-test-slave-49h3"
server_key: "6a80a22e7beefa16070f7df676690e86"
server_key_iv: "f6f0a5d8093fb89b0fb94d8f3d8eb843"
server_key_version: "encryptionkey@0"
I20260808 13:15:46.654419 23219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:46.670262 23219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:46.674069 23219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:46.674355 23219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:46.692399 23219 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:42935
I20260808 13:15:46.692418 23293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:42935 every 8 connection(s)
I20260808 13:15:46.693727 23219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:46.697273 23294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:46.699663 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23219
I20260808 13:15:46.699843 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:46.700376 31982 external_mini_cluster.cc:1428] Setting key 40aa880451c4d03c2d2557dc5c4324ac
I20260808 13:15:46.709512 23294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3: Bootstrap starting.
I20260808 13:15:46.713392 23294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:46.714656 23294 log.cc:824] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:15:46 dist-test-slave-49h3 krb5kdc[23204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:46.718135 23294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3: No bootstrap required, opened a new log
I20260808 13:15:46.724447 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:46.702188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40445 (local address 127.31.59.190:42935)
0808 13:15:46.702944 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:46.702955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:46.702998 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:15:46.704666 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:46.704688 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:46.705072 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:46.705381 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:46.707272 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:46.708603 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:46.710332 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:46.710995 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:46.711139 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:46.716015 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:46.716026 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:46.716046 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:46.716094 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:46.719880 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:46.720565 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:46.720574 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:46.720586 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:46.720804 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:46.721147 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:46.721152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:46.721155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:46.721933 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:46.722249 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:46.722561 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 13:15:46.723021 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":313,"threads_started":1}
I20260808 13:15:46.727957 23294 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } }
I20260808 13:15:46.728272 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:46.728371 23294 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:46.728466 23294 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80c53ab3114c909bd43fb20afa50a3, State: Initialized, Role: FOLLOWER
I20260808 13:15:46.729069 23294 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } }
I20260808 13:15:46.729343 23294 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:46.729486 23294 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:46.729640 23294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:46.732868 23294 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } }
I20260808 13:15:46.733438 23294 leader_election.cc:302] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec80c53ab3114c909bd43fb20afa50a3; no voters:
I20260808 13:15:46.734093 23294 leader_election.cc:288] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:46.734207 23306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:46.736476 23294 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:46.737709 23306 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [term 1 LEADER]: Becoming Leader. State: Replica: ec80c53ab3114c909bd43fb20afa50a3, State: Running, Role: LEADER
I20260808 13:15:46.738391 23306 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } }
I20260808 13:15:46.741485 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec80c53ab3114c909bd43fb20afa50a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } } }
I20260808 13:15:46.741894 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:46.745368 23311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:46.745273 23308 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec80c53ab3114c909bd43fb20afa50a3. Latest consensus state: current_term: 1 leader_uuid: "ec80c53ab3114c909bd43fb20afa50a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec80c53ab3114c909bd43fb20afa50a3" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 42935 } } }
I20260808 13:15:46.745733 23308 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:46.750228 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:46.733289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59737 (local address 127.31.59.190:42935)
0808 13:15:46.733514 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:46.733520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:46.734121 (+ 601us) server_negotiation.cc:407] Connection header received
0808 13:15:46.734303 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:46.734308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:46.734390 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:46.734497 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:46.736257 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:46.737060 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:46.738498 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:46.739013 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:46.739137 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:46.742220 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:46.742227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:46.742232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:46.742281 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:46.746024 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:46.746916 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:46.746921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:46.746924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:46.747078 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:46.747553 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:46.747558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:46.747561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:46.748066 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:46.748414 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:46.749603 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0808 13:15:46.749856 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:15:46.751219 23311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:46.760378 23311 catalog_manager.cc:1347] Generated new cluster ID: 59a08e7762d0407f9dbb47d2fd42f602
I20260808 13:15:46.760546 23311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:46.790155 23311 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:46.791568 23311 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:46.800194 23311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec80c53ab3114c909bd43fb20afa50a3: Generated new TSK 0
I20260808 13:15:46.801028 23311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:46.814173 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23219
I20260808 13:15:46.815258 23289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:46.909413 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23219
2026-08-08T13:15:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2097 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:47 dist-test-slave-49h3 krb5kdc[23330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:47 dist-test-slave-49h3 krb5kdc[23330](info): set up 2 sockets
Aug 08 13:15:47 dist-test-slave-49h3 krb5kdc[23330](info): commencing operation
krb5kdc: starting...
W20260808 13:15:48.004827 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:48 dist-test-slave-49h3 krb5kdc[23330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:48.346482 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45633
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:35715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:45633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:48.683764 23346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:48.684062 23346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:48.684201 23346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:48.691449 23346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:48.691557 23346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:48.691633 23346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:48.691715 23346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:48.691773 23346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:48.701776 23346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:35715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:45633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:45633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:48.703186 23346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:48.704546 23346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:48.716115 23351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:48.716094 23352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:48.716351 23354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:48.716915 23346 server_base.cc:1034] running on GCE node
I20260808 13:15:48.717804 23346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:48.719213 23346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:48.720415 23346 hybrid_clock.cc:648] HybridClock initialized: now 1786194948720371 us; error 38 us; skew 500 ppm
I20260808 13:15:48.720760 23346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:48 dist-test-slave-49h3 krb5kdc[23330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194948, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:48.723834 23346 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:48.725337 23346 webserver.cc:460] Webserver started at http://127.31.59.190:41403/ using document root <none> and password file <none>
I20260808 13:15:48.726085 23346 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:48.726207 23346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:48.726503 23346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:48.729581 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "368d0d73873d451aaf83c372a37b387b"
format_stamp: "Formatted at 2026-08-08 13:15:48 on dist-test-slave-49h3"
server_key: "def85f08c81461159d92fc05b2046ea5"
server_key_iv: "596cf4cd41958b8ec8bbbe36931af480"
server_key_version: "encryptionkey@0"
I20260808 13:15:48.730360 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "368d0d73873d451aaf83c372a37b387b"
format_stamp: "Formatted at 2026-08-08 13:15:48 on dist-test-slave-49h3"
server_key: "def85f08c81461159d92fc05b2046ea5"
server_key_iv: "596cf4cd41958b8ec8bbbe36931af480"
server_key_version: "encryptionkey@0"
I20260808 13:15:48.735529 23346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 13:15:48.738974 23361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:48.740259 23346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:15:48.740427 23346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "368d0d73873d451aaf83c372a37b387b"
format_stamp: "Formatted at 2026-08-08 13:15:48 on dist-test-slave-49h3"
server_key: "def85f08c81461159d92fc05b2046ea5"
server_key_iv: "596cf4cd41958b8ec8bbbe36931af480"
server_key_version: "encryptionkey@0"
I20260808 13:15:48.740609 23346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:48.759884 23346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:48.763844 23346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:48.764140 23346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:48.781817 23346 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:45633
I20260808 13:15:48.781841 23418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:45633 every 8 connection(s)
I20260808 13:15:48.783151 23346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:48.786863 23420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:48.787094 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23346
I20260808 13:15:48.787256 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:48.787667 31982 external_mini_cluster.cc:1428] Setting key f4d27522e23e4b3fb7b8d62f982e448f
I20260808 13:15:48.802225 23420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b: Bootstrap starting.
I20260808 13:15:48.802390 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:48.789359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42057 (local address 127.31.59.190:45633)
0808 13:15:48.790219 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:48.790234 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:48.790285 (+ 51us) server_negotiation.cc:407] Connection header received
0808 13:15:48.791939 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:48.791968 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:48.792520 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:48.792999 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:48.793577 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:48.795555 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:48.796992 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:48.798071 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:48.798273 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:48.799534 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:48.799545 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:48.799569 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:48.800315 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:48.800411 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:48.800449 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:48.800557 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:48.800739 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 13:15:48.801028 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":336,"threads_started":1}
I20260808 13:15:48.805593 23420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:48.806098 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:48.806823 23420 log.cc:824] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:48.810232 23420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b: No bootstrap required, opened a new log
I20260808 13:15:48.817587 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:48.811159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36369 (local address 127.31.59.190:45633)
0808 13:15:48.811332 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:48.811338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:48.811718 (+ 380us) server_negotiation.cc:407] Connection header received
0808 13:15:48.811868 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:48.811877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:48.811961 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:48.812074 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:48.812556 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:48.813716 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:48.815026 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:48.815743 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:48.815878 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:48.816843 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:48.816849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:48.816853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:48.816875 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:48.816923 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:48.816948 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:48.817044 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:48.817130 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 13:15:48.817228 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:15:48.819675 23420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } }
I20260808 13:15:48.820072 23420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:48.820148 23420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 368d0d73873d451aaf83c372a37b387b, State: Initialized, Role: FOLLOWER
I20260808 13:15:48.820708 23420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } }
I20260808 13:15:48.820935 23420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:48.821035 23420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:48.821158 23420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:48.823423 23420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } }
I20260808 13:15:48.823890 23420 leader_election.cc:302] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 368d0d73873d451aaf83c372a37b387b; no voters:
I20260808 13:15:48.824467 23420 leader_election.cc:288] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:48.824633 23433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:48.825769 23433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [term 1 LEADER]: Becoming Leader. State: Replica: 368d0d73873d451aaf83c372a37b387b, State: Running, Role: LEADER
I20260808 13:15:48.826333 23433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } }
I20260808 13:15:48.826731 23420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:48.829092 23437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 368d0d73873d451aaf83c372a37b387b. Latest consensus state: current_term: 1 leader_uuid: "368d0d73873d451aaf83c372a37b387b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } } }
I20260808 13:15:48.829538 23437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:48.829643 23435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "368d0d73873d451aaf83c372a37b387b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d0d73873d451aaf83c372a37b387b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 45633 } } }
I20260808 13:15:48.829932 23435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:48.834177 23442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:48.839442 23442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:48.850136 23442 catalog_manager.cc:1347] Generated new cluster ID: 7c4de00d02344e84baa9226cb4fcf720
I20260808 13:15:48.850561 23442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:48.891559 23442 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:48.893054 23442 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:48.904026 23442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 368d0d73873d451aaf83c372a37b387b: Generated new TSK 0
I20260808 13:15:48.905146 23442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:48.915154 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23346
W20260808 13:15:48.916369 23413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42057
W20260808 13:15:48.917189 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:48.917328 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23346
2026-08-08T13:15:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2007 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:49 dist-test-slave-49h3 krb5kdc[23456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:49 dist-test-slave-49h3 krb5kdc[23456](info): set up 2 sockets
Aug 08 13:15:49 dist-test-slave-49h3 krb5kdc[23456](info): commencing operation
krb5kdc: starting...
W20260808 13:15:50.025383 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:50 dist-test-slave-49h3 krb5kdc[23456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:50.361124 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33411
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:33411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:50.697439 23470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:50.697764 23470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:50.697907 23470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:50.705482 23470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:50.705602 23470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:50.705680 23470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:50.705760 23470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:50.705808 23470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:50.716199 23470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:33411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:33411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:50.717593 23470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:50.718905 23470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:50.730543 23479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:50.730620 23476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:50.730593 23477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:50.731472 23470 server_base.cc:1034] running on GCE node
I20260808 13:15:50.732209 23470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:50.733594 23470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:50.734805 23470 hybrid_clock.cc:648] HybridClock initialized: now 1786194950734759 us; error 39 us; skew 500 ppm
I20260808 13:15:50.735141 23470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:50 dist-test-slave-49h3 krb5kdc[23456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194950, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:50.743314 23470 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:50.744820 23470 webserver.cc:460] Webserver started at http://127.31.59.190:42811/ using document root <none> and password file <none>
I20260808 13:15:50.745558 23470 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:50.745689 23470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:50.746002 23470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:50.749075 23470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "0c764a7d9cc644389a19c0b906a7eda9"
format_stamp: "Formatted at 2026-08-08 13:15:50 on dist-test-slave-49h3"
server_key: "5a848ffe62ab2c4b09cbcc56a77b704b"
server_key_iv: "7ee4a5a15089002eee05853fa6a81422"
server_key_version: "encryptionkey@0"
I20260808 13:15:50.749864 23470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "0c764a7d9cc644389a19c0b906a7eda9"
format_stamp: "Formatted at 2026-08-08 13:15:50 on dist-test-slave-49h3"
server_key: "5a848ffe62ab2c4b09cbcc56a77b704b"
server_key_iv: "7ee4a5a15089002eee05853fa6a81422"
server_key_version: "encryptionkey@0"
I20260808 13:15:50.754944 23470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:15:50.758415 23486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:50.759699 23470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:15:50.759872 23470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "0c764a7d9cc644389a19c0b906a7eda9"
format_stamp: "Formatted at 2026-08-08 13:15:50 on dist-test-slave-49h3"
server_key: "5a848ffe62ab2c4b09cbcc56a77b704b"
server_key_iv: "7ee4a5a15089002eee05853fa6a81422"
server_key_version: "encryptionkey@0"
I20260808 13:15:50.760058 23470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:50.765687 23470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:50.769506 23470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:50.769783 23470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:50.787408 23470 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:33411
I20260808 13:15:50.787427 23540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:33411 every 8 connection(s)
I20260808 13:15:50.788836 23470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:50.792603 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:50.793100 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23470
I20260808 13:15:50.793291 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:50.793883 31982 external_mini_cluster.cc:1428] Setting key 70aea5d44881066123e1e67c8d515a61
I20260808 13:15:50.802157 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:50.795633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43591 (local address 127.31.59.190:33411)
0808 13:15:50.796458 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:50.796469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:50.796512 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:15:50.798832 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:50.798855 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:50.799227 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:50.799573 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:50.800618 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:50.800625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:50.800637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:50.800652 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:50.800713 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:50.800732 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:50.800899 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:50.800998 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 13:15:50.801173 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":380,"threads_started":1}
I20260808 13:15:50.805646 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:50.805737 23541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9: Bootstrap starting.
I20260808 13:15:50.809535 23541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:50.810863 23541 log.cc:824] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:50.813396 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:50.810790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45029 (local address 127.31.59.190:33411)
0808 13:15:50.810975 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:50.810982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:50.811603 (+ 621us) server_negotiation.cc:407] Connection header received
0808 13:15:50.811771 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:50.811775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:50.811828 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:50.811914 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:50.812795 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:50.812798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:50.812802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:15:50.812817 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:50.812855 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:15:50.812869 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:50.812963 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:50.813079 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 13:15:50.813138 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:15:50.813688 23541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9: No bootstrap required, opened a new log
I20260808 13:15:50.821974 23541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } }
I20260808 13:15:50.822357 23541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:15:50.822453 23541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c764a7d9cc644389a19c0b906a7eda9, State: Initialized, Role: FOLLOWER
I20260808 13:15:50.823014 23541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } }
I20260808 13:15:50.823246 23541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:15:50.823375 23541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:15:50.823524 23541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:50.825994 23541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } }
I20260808 13:15:50.826558 23541 leader_election.cc:302] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c764a7d9cc644389a19c0b906a7eda9; no voters:
I20260808 13:15:50.827183 23541 leader_election.cc:288] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:15:50.827327 23551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:50.828001 23551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [term 1 LEADER]: Becoming Leader. State: Replica: 0c764a7d9cc644389a19c0b906a7eda9, State: Running, Role: LEADER
I20260808 13:15:50.828547 23551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } }
I20260808 13:15:50.829203 23541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:50.831367 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c764a7d9cc644389a19c0b906a7eda9. Latest consensus state: current_term: 1 leader_uuid: "0c764a7d9cc644389a19c0b906a7eda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } } }
I20260808 13:15:50.831881 23552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c764a7d9cc644389a19c0b906a7eda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c764a7d9cc644389a19c0b906a7eda9" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 33411 } } }
I20260808 13:15:50.832199 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:50.832286 23552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9 [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:50.834734 23561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:50.838917 23561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:15:50.847538 23561 catalog_manager.cc:1347] Generated new cluster ID: cfc5e8a30f4b45058c3220ce9dd85d74
I20260808 13:15:50.847676 23561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:50.862468 23561 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:50.862900 23561 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:50.870131 23561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c764a7d9cc644389a19c0b906a7eda9: Generated new TSK 0
I20260808 13:15:50.870965 23561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:50.877918 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23470
W20260808 13:15:50.879051 23536 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43591
W20260808 13:15:50.879756 31982 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23470: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:15:50.879931 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23470
2026-08-08T13:15:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1963 ms)
[----------] 18 tests from AuthTokenIssuingTest (36200 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:50 dist-test-slave-49h3 krb5kdc[23575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:50 dist-test-slave-49h3 krb5kdc[23575](info): set up 2 sockets
Aug 08 13:15:50 dist-test-slave-49h3 krb5kdc[23575](info): commencing operation
krb5kdc: starting...
W20260808 13:15:51.975821 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:52 dist-test-slave-49h3 krb5kdc[23575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:52.313176 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39069
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:52.651618 23589 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:52.651958 23589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:52.652031 23589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:52.652114 23589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:52.659461 23589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:52.659566 23589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:52.659626 23589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:52.659672 23589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:52.659729 23589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:52.669872 23589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:39069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:52.671185 23589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:52.672547 23589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:52.684196 23597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:52.684345 23600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:52.684551 23596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:52.685227 23589 server_base.cc:1034] running on GCE node
I20260808 13:15:52.686026 23589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:52.687417 23589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:52.688638 23589 hybrid_clock.cc:648] HybridClock initialized: now 1786194952688569 us; error 45 us; skew 500 ppm
I20260808 13:15:52.688968 23589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:52 dist-test-slave-49h3 krb5kdc[23575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:52.692088 23589 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:52.693629 23589 webserver.cc:460] Webserver started at http://127.31.59.190:43655/ using document root <none> and password file <none>
I20260808 13:15:52.694315 23589 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:52.694417 23589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:52.694730 23589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:52.697777 23589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "2d470cf671f24aa28e1a152fb4add13b"
format_stamp: "Formatted at 2026-08-08 13:15:52 on dist-test-slave-49h3"
server_key: "69dbb1e64c4d9f301283917baa24e532"
server_key_iv: "d16af3bf73290cb245726752c94f7114"
server_key_version: "encryptionkey@0"
I20260808 13:15:52.698575 23589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "2d470cf671f24aa28e1a152fb4add13b"
format_stamp: "Formatted at 2026-08-08 13:15:52 on dist-test-slave-49h3"
server_key: "69dbb1e64c4d9f301283917baa24e532"
server_key_iv: "d16af3bf73290cb245726752c94f7114"
server_key_version: "encryptionkey@0"
I20260808 13:15:52.703840 23589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:52.707405 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:52.708756 23589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:52.708925 23589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "2d470cf671f24aa28e1a152fb4add13b"
format_stamp: "Formatted at 2026-08-08 13:15:52 on dist-test-slave-49h3"
server_key: "69dbb1e64c4d9f301283917baa24e532"
server_key_iv: "d16af3bf73290cb245726752c94f7114"
server_key_version: "encryptionkey@0"
I20260808 13:15:52.709138 23589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:52.726569 23589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:52.730749 23589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:52.731022 23589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:52.749264 23589 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:39069
I20260808 13:15:52.749281 23666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:39069 every 8 connection(s)
I20260808 13:15:52.750643 23589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:52.754344 23667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:52.755553 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23589
I20260808 13:15:52.755812 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:52.756675 31982 external_mini_cluster.cc:1428] Setting key 43f19bcc6667b51a38a9bb51800ecf18
I20260808 13:15:52.763495 23667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:52 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:52.787819 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.767274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39069 (local address 127.0.0.1:37380)
0808 13:15:52.768198 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.768256 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:52.770609 (+ 2353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:52.771465 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:52.771476 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:52.772247 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:52.772722 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:52.772747 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:52.774351 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:52.774355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:52.775497 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:52.775510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:52.776745 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:52.776751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:52.776885 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:52.777547 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:52.777569 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:52.779799 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:52.783746 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:52.783757 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:52.783768 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:52.784169 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:52.784666 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:52.784670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:52.784673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:52.784872 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:52.785837 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:52.785847 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:52.786060 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 13:15:52.786292 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 13:15:52.786706 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":390,"threads_started":1}
I20260808 13:15:52.787871 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.767274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.31.59.190:39069)
0808 13:15:52.769073 (+ 1799us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:52.769079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:52.769127 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:15:52.770986 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:52.770992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:52.771228 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:52.771400 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:52.772932 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:52.774190 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:52.775643 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:52.776493 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:52.776659 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:52.780046 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:52.780056 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:52.780071 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:52.780111 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:52.783604 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:52.784308 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:52.784316 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:52.784319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:52.784529 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:52.784987 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:52.784993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:52.784995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:52.785696 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:52.785892 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:52.786482 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 13:15:52.786879 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":1598,"spinlock_wait_cycles":25856,"thread_start_us":1368,"threads_started":1}
I20260808 13:15:52.790781 23667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:52.791953 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.791530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55128)
0808 13:15:52.791737 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.791840 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 13:15:52.792222 23609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.59.189@KRBTEST.COM; defaulting to no policy
I20260808 13:15:52.793814 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.793408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55136)
0808 13:15:52.793585 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.793665 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 13:15:52.793996 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:35563: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
I20260808 13:15:52.832790 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:35563
--webserver_interface=127.31.59.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:15:52.843340 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } attempt: 1
I20260808 13:15:52.844523 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.844065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55148)
0808 13:15:52.844266 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.844382 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 13:15:52.845542 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.845155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55156)
0808 13:15:52.845306 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.845378 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 13:15:52.845746 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:35563: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
I20260808 13:15:52.913995 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } attempt: 2
I20260808 13:15:52.915249 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.914659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55170)
0808 13:15:52.914943 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.915065 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 13:15:52.916337 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:52.915918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55186)
0808 13:15:52.916069 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:52.916161 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 13:15:52.916550 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:35563: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
I20260808 13:15:53.007795 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } attempt: 3
I20260808 13:15:53.008948 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.008528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55188)
0808 13:15:53.008718 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.008812 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 13:15:53.009976 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.009615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55200)
0808 13:15:53.009765 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.009835 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 13:15:53.010174 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:35563: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
I20260808 13:15:53.153417 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } attempt: 4
I20260808 13:15:53.154749 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.154058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55216)
0808 13:15:53.154468 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.154602 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260808 13:15:53.155877 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.155499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55232)
0808 13:15:53.155656 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.155745 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 13:15:53.156092 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:35563: Network error: Client connection negotiation failed: client connection to 127.31.59.189:35563: connect: Connection refused (error 111)
W20260808 13:15:53.186175 23675 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:53.186595 23675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:53.186699 23675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:53.186816 23675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:53.194458 23675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:53.194569 23675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:53.194674 23675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:53.194743 23675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:53.194831 23675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:53.205710 23675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:35563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:53.207482 23675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:53.208868 23675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:53.220665 23684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:53.220721 23680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:53.220755 23681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:53.221761 23675 server_base.cc:1034] running on GCE node
I20260808 13:15:53.222560 23675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:53.223917 23675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:53.225133 23675 hybrid_clock.cc:648] HybridClock initialized: now 1786194953225062 us; error 38 us; skew 500 ppm
I20260808 13:15:53.225517 23675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:53.228652 23675 init.cc:377] Logged in from keytab as kudu/127.31.59.189@KRBTEST.COM (short username kudu)
I20260808 13:15:53.230156 23675 webserver.cc:460] Webserver started at http://127.31.59.189:45009/ using document root <none> and password file <none>
I20260808 13:15:53.230835 23675 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:53.230945 23675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:53.231264 23675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:53.234409 23675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data/instance:
uuid: "ecf325b2cddc4a44bf6aa1f92ae53468"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "aafc43af8c887c845a5de055eaf98a2e"
server_key_iv: "6f9fb5b0bfef0a7afa1a94778366c3b2"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.235252 23675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal/instance:
uuid: "ecf325b2cddc4a44bf6aa1f92ae53468"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "aafc43af8c887c845a5de055eaf98a2e"
server_key_iv: "6f9fb5b0bfef0a7afa1a94778366c3b2"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.240499 23675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:15:53.244035 23691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:53.245342 23675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:53.245528 23675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "ecf325b2cddc4a44bf6aa1f92ae53468"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "aafc43af8c887c845a5de055eaf98a2e"
server_key_iv: "6f9fb5b0bfef0a7afa1a94778366c3b2"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.245734 23675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:53.262898 23675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:53.267247 23675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:53.267504 23675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:53.285589 23675 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.189:35563
I20260808 13:15:53.285595 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.189:35563 every 8 connection(s)
I20260808 13:15:53.286927 23675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
I20260808 13:15:53.290518 23755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:53.296311 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23675
I20260808 13:15:53.296515 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-1/wal/instance
I20260808 13:15:53.297073 31982 external_mini_cluster.cc:1428] Setting key 80d66985a6a256ae7077ca7fc0d3a004
I20260808 13:15:53.297747 23755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:53.322294 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.300888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.31.59.190:39069)
0808 13:15:53.301409 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.301416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.302417 (+ 1001us) server_negotiation.cc:407] Connection header received
0808 13:15:53.304740 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.304745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.304824 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.304914 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.306930 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.307653 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.309706 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.310131 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.310238 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.314462 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.314467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.314470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.314551 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.317832 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.318892 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.318896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.318899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.319072 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.319674 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.319680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.319683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.320076 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.320231 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.321353 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.321891 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":245,"threads_started":1}
I20260808 13:15:53.323172 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.301377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39069 (local address 127.0.0.1:37390)
0808 13:15:53.302225 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.302281 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.304445 (+ 2164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.305167 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.305177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.305902 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.306689 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.306715 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.307822 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.307828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.309580 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.309595 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.310359 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.310365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.310633 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.311335 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.311364 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.314175 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.318024 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.318035 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.318054 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.318695 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.319204 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.319209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.319212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.319521 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.320274 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.320282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.320822 (+ 540us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.321109 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.321730 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":317,"threads_started":1}
I20260808 13:15:53.325598 23755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.59.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.188@KRBTEST.COM" created.
I20260808 13:15:53.341248 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.326444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55238)
0808 13:15:53.326662 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.326683 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.326805 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.328180 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.328185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.328644 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.328825 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.328836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.330287 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.330292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.331549 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.331564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.332467 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.332471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.332595 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.333211 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.333239 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.335464 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.338858 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.338863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.338866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.339240 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.339596 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.339600 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.339602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.339730 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.340313 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.340318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.340456 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.340570 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.340841 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:15:53.341327 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.327031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.31.59.189:35563)
0808 13:15:53.327743 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.327756 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.327784 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:15:53.327827 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.327833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.328009 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.328121 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.328995 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.330139 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.331702 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.332315 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.332416 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.335687 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.335703 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.335722 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.335766 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.338736 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.339363 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.339369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.339371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.339493 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.339839 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.339842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.339844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.340188 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.340395 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.340750 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.340967 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":405,"threads_started":1}
I20260808 13:15:53.343622 23755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:53.344811 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.344365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44690)
0808 13:15:53.344542 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.344655 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 13:15:53.345100 23694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.346747 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.346412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44700)
0808 13:15:53.346548 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.346631 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 13:15:53.346987 23755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
I20260808 13:15:53.376801 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:33279
--webserver_interface=127.31.59.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:15:53.396353 23755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 1
I20260808 13:15:53.397630 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.397124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44706)
0808 13:15:53.397317 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.397428 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 13:15:53.398625 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.398270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44722)
0808 13:15:53.398431 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.398512 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:15:53.398839 23755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.455372 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } attempt: 5
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:53.467039 23755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 2
I20260808 13:15:53.467947 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.467531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44730)
0808 13:15:53.467706 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.467793 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 13:15:53.469096 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.468714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44732)
0808 13:15:53.468856 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.468945 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 13:15:53.469260 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.456290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.31.59.189:35563)
0808 13:15:53.456467 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.456473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.456495 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:15:53.456670 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.456674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.456790 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.456889 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.457817 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.458507 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.459725 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.460166 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.460278 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.463597 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.463602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.463605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.463639 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.466317 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.466956 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.466961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.466963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.467125 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.467698 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.467704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.467707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.468119 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.468288 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.468723 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.468974 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 13:15:53.469628 23755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.469776 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.456163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55240)
0808 13:15:53.456379 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.456399 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.456536 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.456946 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.456951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.457406 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.457653 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.457665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.458649 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.458652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.459579 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.459593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.460322 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.460328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.460539 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.461157 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.461174 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.463391 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.466450 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.466456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.466460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.466845 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.467345 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.467351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.467355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.467542 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.468266 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.468271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.468445 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.468611 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.469463 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 13:15:53.470824 23667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:53.471823 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.471456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44740)
0808 13:15:53.471617 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.471688 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 13:15:53.472708 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.472402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44750)
0808 13:15:53.472531 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.472599 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 13:15:53.472914 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.524134 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 1
I20260808 13:15:53.525355 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.524865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44754)
0808 13:15:53.525073 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.525166 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 13:15:53.526319 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.525971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44766)
0808 13:15:53.526129 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.526202 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 13:15:53.526521 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.560883 23755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 3
I20260808 13:15:53.562140 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.561651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44780)
0808 13:15:53.561876 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.561980 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 13:15:53.563297 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.562842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44788)
0808 13:15:53.563085 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.563172 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260808 13:15:53.563524 23755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.594713 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 2
I20260808 13:15:53.595914 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.595421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44798)
0808 13:15:53.595641 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.595744 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 13:15:53.597054 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.596671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44812)
0808 13:15:53.596821 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.596910 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:15:53.597292 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.703503 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 3
I20260808 13:15:53.704667 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.704201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44814)
0808 13:15:53.704410 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.704502 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 13:15:53.705710 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.705317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44816)
0808 13:15:53.705537 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.705610 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260808 13:15:53.705914 23667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
I20260808 13:15:53.706717 23755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 4
I20260808 13:15:53.707666 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.707272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44828)
0808 13:15:53.707455 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.707532 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 13:15:53.708576 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.708286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44834)
0808 13:15:53.708411 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.708477 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:15:53.708796 23755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:33279: Network error: Client connection negotiation failed: client connection to 127.31.59.188:33279: connect: Connection refused (error 111)
W20260808 13:15:53.725045 23764 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:53.725382 23764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:53.725482 23764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:53.725579 23764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:53.733163 23764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:53.733266 23764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:53.733326 23764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:53.733402 23764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:53.733511 23764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:53.743574 23764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:39069,127.31.59.189:35563,127.31.59.188:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:53.744892 23764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:53.746203 23764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:53.759373 23769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:53.759716 23770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:53.759742 23772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:53.760782 23764 server_base.cc:1034] running on GCE node
I20260808 13:15:53.761502 23764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:53.762830 23764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:53.764035 23764 hybrid_clock.cc:648] HybridClock initialized: now 1786194953763965 us; error 49 us; skew 500 ppm
I20260808 13:15:53.764331 23764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:53.767469 23764 init.cc:377] Logged in from keytab as kudu/127.31.59.188@KRBTEST.COM (short username kudu)
I20260808 13:15:53.768929 23764 webserver.cc:460] Webserver started at http://127.31.59.188:41681/ using document root <none> and password file <none>
I20260808 13:15:53.769647 23764 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:53.769742 23764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:53.770031 23764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:53.773061 23764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data/instance:
uuid: "d3d5332710fb4bb2931f2346b2684cf5"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "d87b5bd1f8efb1f00b769f4ec602f969"
server_key_iv: "cc2cfd108d8cf28230b243cf8f030591"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.773874 23764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal/instance:
uuid: "d3d5332710fb4bb2931f2346b2684cf5"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "d87b5bd1f8efb1f00b769f4ec602f969"
server_key_iv: "cc2cfd108d8cf28230b243cf8f030591"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.778764 23764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:15:53.782044 23779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:53.783283 23764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:15:53.783424 23764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "d3d5332710fb4bb2931f2346b2684cf5"
format_stamp: "Formatted at 2026-08-08 13:15:53 on dist-test-slave-49h3"
server_key: "d87b5bd1f8efb1f00b769f4ec602f969"
server_key_iv: "cc2cfd108d8cf28230b243cf8f030591"
server_key_version: "encryptionkey@0"
I20260808 13:15:53.783601 23764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:53.805866 23764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:53.810122 23764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:53.810352 23764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:53.827836 23764 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.188:33279
I20260808 13:15:53.827862 23836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.188:33279 every 8 connection(s)
I20260808 13:15:53.829151 23764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
I20260808 13:15:53.831636 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 23764
I20260808 13:15:53.831816 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194589892156-31982-0/minicluster-data/master-2/wal/instance
I20260808 13:15:53.832233 31982 external_mini_cluster.cc:1428] Setting key f25171fbd2c59bda215cb564ec28d343
I20260808 13:15:53.832931 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:53.842525 23838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:53.855269 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.834016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.31.59.188:33279)
0808 13:15:53.834786 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.834797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.834838 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:15:53.836363 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.836384 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.836746 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.837058 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.838893 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.840193 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.841960 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.842638 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.842771 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.847718 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.847725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.847734 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.847771 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.851346 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.851991 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.852000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.852009 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.852189 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.852569 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.852574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.852578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.853088 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.853418 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.853846 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.854264 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":356,"threads_started":1}
I20260808 13:15:53.860369 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.845578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:39069 (local address 127.0.0.1:37400)
0808 13:15:53.846528 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.846576 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.846756 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.847133 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.847140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.847940 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.848236 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.848247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.849320 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.849325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.851119 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.851136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.851874 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.851881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.852049 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.852577 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.852599 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.854710 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.857700 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.857708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.857713 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.858278 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.858660 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.858665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.858668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.858797 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.859306 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.859313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.859479 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.859626 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.859959 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":527,"threads_started":1}
I20260808 13:15:53.860383 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.845579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.31.59.190:39069)
0808 13:15:53.846123 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.846129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.846678 (+ 549us) server_negotiation.cc:407] Connection header received
0808 13:15:53.846867 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.846871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.846952 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.847037 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.848385 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.849170 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.851285 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.851685 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.851808 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.854898 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.854903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.854906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.854947 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.857574 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.858403 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.858409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.858413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.858555 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.858917 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.858922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.858926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.859183 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.859349 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.859780 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.859991 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":253,"threads_started":1}
I20260808 13:15:53.862377 23838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:53.875449 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.858154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.31.59.190:39069)
0808 13:15:53.860598 (+ 2444us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.860603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.860619 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:15:53.860675 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.860679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.860773 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.860865 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.862114 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.863008 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.864306 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.864767 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.864858 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.869493 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.869502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.869507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.869550 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.872683 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.873341 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.873346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.873349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.873541 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.873929 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.873932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.873934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.874278 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.874530 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.874869 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.875105 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2344,"thread_start_us":2833,"threads_started":1}
I20260808 13:15:53.876063 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.863167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:35563 (local address 127.0.0.1:55246)
0808 13:15:53.863339 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.863358 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.863460 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.863752 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.863757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.864110 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.864255 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.864266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.865353 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.865358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.866363 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.866375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.867302 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.867306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.867440 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.867917 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.867933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.870031 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.873433 (+ 3402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.873437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.873439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.873974 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.874380 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.874384 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.874387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.874565 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.875250 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.875255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.875430 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.875578 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.875831 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 13:15:53.876338 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.863174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.31.59.189:35563)
0808 13:15:53.863366 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.863371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.863406 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:15:53.863566 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.863571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.863635 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.863713 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.864404 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.865198 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.866514 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.867152 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.867304 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.870249 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.870256 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.870260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.870306 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.873293 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.874087 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.874093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.874096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.874275 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.874684 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.874688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.874690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.875023 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.875171 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.875720 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.875950 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:15:53.877696 23838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:53.883107 23667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 4
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
Aug 08 13:15:53 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194952, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:15:53.892249 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.878380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44846)
0808 13:15:53.878550 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.878568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.878664 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.878922 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.878927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.879394 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.879533 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.879541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.880777 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.880781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.881676 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.881688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.882575 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.882581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.882796 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.883612 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.883634 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.886158 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.889910 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.889915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.889918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.890335 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.890667 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.890672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.890676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.890801 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.891278 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.891282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.891416 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.891563 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.891856 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 13:15:53.892769 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.878535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.31.59.188:33279)
0808 13:15:53.878672 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.878676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.878690 (+ 14us) server_negotiation.cc:407] Connection header received
0808 13:15:53.878752 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.878755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.878804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.878873 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.879660 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.880645 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.881797 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.882413 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.882536 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.886330 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.886335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.886338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.886366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.889773 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.890439 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.890444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.890448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.890570 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.890896 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.890901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.890904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.891163 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.891290 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.892159 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.892433 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 13:15:53.895494 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.877330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.31.59.189:35563)
0808 13:15:53.877524 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.877530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.877581 (+ 51us) server_negotiation.cc:407] Connection header received
0808 13:15:53.877741 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.877746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.877805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.877884 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.879390 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.880461 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.881874 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.882501 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.882609 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.888205 (+ 5596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.888211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.888214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.888253 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.891352 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.892182 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.892188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.892191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.892364 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.892790 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.892796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.892800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.893565 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.893827 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.894941 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.895189 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 13:15:53.898921 23838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: Bootstrap starting.
I20260808 13:15:53.899657 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.883700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44858)
0808 13:15:53.883878 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:53.883896 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:53.883984 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:53.884873 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:53.884878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:53.885379 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:53.885610 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.885624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.887143 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.887149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.888436 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:53.888452 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.889442 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.889470 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:53.889683 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.890204 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:53.890230 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:53.893052 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:53.897045 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.897050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.897055 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.897487 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.897934 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:53.897940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:53.897943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:53.898113 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:53.898688 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:53.898693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:53.898856 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 13:15:53.898986 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 13:15:53.899276 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 13:15:53.900867 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:53.900756 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.883764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.31.59.188:33279)
0808 13:15:53.884528 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.884534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.884558 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:15:53.884611 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.884616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.884686 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.884808 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.885791 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.886987 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.888631 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.889278 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.889412 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.893254 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.893261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.893265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.893300 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.896921 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.897628 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.897633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.897636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.897796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.898243 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.898247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.898249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.898515 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.898666 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.899175 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.899390 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":352,"threads_started":1}
I20260808 13:15:53.903102 23838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:53.904520 23838 log.cc:824] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:53.904366 23639 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d470cf671f24aa28e1a152fb4add13b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37416
I20260808 13:15:53.909155 23667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: Bootstrap starting.
I20260808 13:15:53.910560 23838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: No bootstrap required, opened a new log
I20260808 13:15:53.913045 23667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:53.914322 23667 log.cc:824] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:53.917565 23667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: No bootstrap required, opened a new log
I20260808 13:15:53.920535 23838 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.920954 23838 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d5332710fb4bb2931f2346b2684cf5, State: Initialized, Role: FOLLOWER
I20260808 13:15:53.921471 23838 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.922435 23849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:53.922861 23849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:53.923629 23838 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:53.924664 23667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.925073 23667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d470cf671f24aa28e1a152fb4add13b, State: Initialized, Role: FOLLOWER
I20260808 13:15:53.925591 23667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.926613 23857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:53.927008 23857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:53.927799 23667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:53.927868 23639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 13:15:53.928056 23639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:53.930539 23639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.932526 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d470cf671f24aa28e1a152fb4add13b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d3d5332710fb4bb2931f2346b2684cf5"
I20260808 13:15:53.933014 23809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:53.936573 23809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d470cf671f24aa28e1a152fb4add13b in term 1.
I20260808 13:15:53.937268 23611 leader_election.cc:302] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d470cf671f24aa28e1a152fb4add13b, d3d5332710fb4bb2931f2346b2684cf5; no voters:
I20260808 13:15:53.937770 23639 leader_election.cc:288] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [CANDIDATE]: Term 1 election: Requested vote from peers ecf325b2cddc4a44bf6aa1f92ae53468 (127.31.59.189:35563), d3d5332710fb4bb2931f2346b2684cf5 (127.31.59.188:33279)
I20260808 13:15:53.939821 23857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:53.945179 23857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [term 1 LEADER]: Becoming Leader. State: Replica: 2d470cf671f24aa28e1a152fb4add13b, State: Running, Role: LEADER
I20260808 13:15:53.945921 23857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:53.949980 23869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d470cf671f24aa28e1a152fb4add13b. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:53.950419 23869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:53.951851 23880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:53.960412 23880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 13:15:53.963217 23875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:15:53.963382 23875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:15:53.950752 23764 master_runner.cc:186] Error getting master registration for 127.31.59.190:39069: OK, instance_id { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" instance_seqno: 1786194952744119 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:15:53.964277 23764 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:15:53.983932 23615 tablet.cc:2369] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:53.993885 23891 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:15:53.995787 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.966698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.31.59.189:35563)
0808 13:15:53.967076 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.967082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.976207 (+ 9125us) server_negotiation.cc:407] Connection header received
0808 13:15:53.976747 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.976755 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.976849 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.976925 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.979233 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.979976 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.981872 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.982551 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.982650 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.986503 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.986509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.986513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.986557 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.990329 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.991723 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.991730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.991733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.991994 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.992717 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.992722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.992725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.993302 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:53.993693 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:53.994681 (+ 988us) server_negotiation.cc:299] Negotiation successful
0808 13:15:53.995064 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260808 13:15:54.000357 23809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [term 1 FOLLOWER]: Refusing update from remote peer 2d470cf671f24aa28e1a152fb4add13b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:54.001529 23857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:54.002019 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.966698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.31.59.190:39069)
0808 13:15:53.966919 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.966925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.969860 (+ 2935us) server_negotiation.cc:407] Connection header received
0808 13:15:53.971248 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.971254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.971344 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.971421 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.981357 (+ 9936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.982538 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.985484 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.986222 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.986346 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:53.992680 (+ 6334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:53.992688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:53.992693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:53.992738 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:53.998257 (+ 5519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.999179 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.999185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.999189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:53.999365 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:53.999835 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:53.999840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:53.999843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:54.000274 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:54.000610 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:54.001182 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 13:15:54.001435 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:15:54.007956 23755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } attempt: 5
I20260808 13:15:54.003216 23589 master_runner.cc:186] Error getting master registration for 127.31.59.189:35563: OK, instance_id { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" instance_seqno: 1786194953280413 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Aug 08 13:15:54 dist-test-slave-49h3 krb5kdc[23575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194953, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:15:54.019286 23589 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:15:54.024371 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:53.968536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.31.59.188:33279)
0808 13:15:53.968721 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:53.968727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:53.977214 (+ 8487us) server_negotiation.cc:407] Connection header received
0808 13:15:53.978164 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:53.978170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:53.978280 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:53.978355 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:53.980879 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.981968 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.998302 (+ 16334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:53.998940 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:53.999053 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:54.013482 (+ 14429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:54.013489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:54.013494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:54.013538 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:54.020166 (+ 6628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:54.021052 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:54.021057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:54.021061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:54.021227 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:54.021782 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:54.021787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:54.021791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:54.022181 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:54.022429 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:54.023034 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 13:15:54.023412 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:15:54.032784 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:54.008490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:33279 (local address 127.0.0.1:44868)
0808 13:15:54.008901 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:54.008920 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:54.009030 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:54.009917 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:54.009923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:54.010483 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:54.010705 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:54.010717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:54.012248 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:54.012255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:54.013815 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:54.013833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:54.015310 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:54.015317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:54.015578 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:54.016424 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:54.016448 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:54.019697 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:54.028876 (+ 9179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:54.028886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:54.028890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:54.029526 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:54.030274 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:54.030280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:54.030285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:54.030472 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:54.031671 (+ 1199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:54.031677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:54.031859 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 13:15:54.032026 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 13:15:54.032367 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287}
I20260808 13:15:54.033700 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:54.009235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44868 (local address 127.31.59.188:33279)
0808 13:15:54.009429 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.009434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.009482 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:15:54.009542 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.009547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.009629 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.009724 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.010886 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:54.012069 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:54.014000 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:54.014780 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:54.014918 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:54.020004 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:54.020011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:54.020017 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:54.020064 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:54.028553 (+ 8489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:54.029698 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:54.029704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:54.029708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:54.030073 (+ 365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:54.030632 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:54.030639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:54.030643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:54.031485 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:54.031839 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:54.032316 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 13:15:54.032811 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:15:54.035527 23849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d470cf671f24aa28e1a152fb4add13b. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.035890 23849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:54.038998 23892 mvcc.cc:205] Tried to move back new op lower bound from 7316254531570860032 to 7316254531367886848. Current Snapshot: MvccSnapshot[applied={T|T < 7316254531570860032}]
I20260808 13:15:54.040652 23755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: Bootstrap starting.
I20260808 13:15:54.046164 23893 mvcc.cc:205] Tried to move back new op lower bound from 7316254531570860032 to 7316254531367886848. Current Snapshot: MvccSnapshot[applied={T|T < 7316254531570860032}]
I20260808 13:15:54.050676 23755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:54.053655 23755 log.cc:824] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:54.056126 23869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.056443 23869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:54.058499 23880 catalog_manager.cc:1347] Generated new cluster ID: 8d5b711b72c5449897583b8cd607591f
I20260808 13:15:54.058624 23880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:54.060719 23849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.060984 23849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:54.066517 23755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: No bootstrap required, opened a new log
I20260808 13:15:54.077318 23755 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:54.077754 23755 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf325b2cddc4a44bf6aa1f92ae53468, State: Initialized, Role: FOLLOWER
I20260808 13:15:54.078253 23755 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } }
I20260808 13:15:54.085175 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.085572 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:54.087508 23755 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:54.087546 23724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d470cf671f24aa28e1a152fb4add13b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468"
I20260808 13:15:54.087947 23723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:54.092136 23723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [term 1 FOLLOWER]: Refusing update from remote peer 2d470cf671f24aa28e1a152fb4add13b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:54.095518 23724 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d470cf671f24aa28e1a152fb4add13b in term 1.
I20260808 13:15:54.096001 23857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:54.114439 23880 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:54.116477 23880 catalog_manager.cc:1504] Loading token signing keys...
W20260808 13:15:54.135157 23909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:15:54.135437 23909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:15:54.167078 23916 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.167536 23916 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:54.167701 23914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.168157 23914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:54.168087 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d470cf671f24aa28e1a152fb4add13b. Latest consensus state: current_term: 1 leader_uuid: "2d470cf671f24aa28e1a152fb4add13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d470cf671f24aa28e1a152fb4add13b" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 39069 } } peers { permanent_uuid: "ecf325b2cddc4a44bf6aa1f92ae53468" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 35563 } } peers { permanent_uuid: "d3d5332710fb4bb2931f2346b2684cf5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 33279 } } }
I20260808 13:15:54.168387 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:54.178694 23880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d470cf671f24aa28e1a152fb4add13b: Generated new TSK 0
I20260808 13:15:54.179811 23880 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:15:54.328083 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.319277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.31.59.189:35563)
0808 13:15:54.319685 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.319690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.324661 (+ 4971us) server_negotiation.cc:407] Connection header received
0808 13:15:54.325191 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.325198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.325295 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.325367 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.327828 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55274: BlockingRecv error: recv got EOF from 127.0.0.1:55274 (error 108)
Metrics: {"server-negotiator.queue_time_us":247}
W20260808 13:15:54.331502 23842 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.319017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.31.59.190:39069)
0808 13:15:54.319495 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.319501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.328665 (+ 9164us) server_negotiation.cc:407] Connection header received
0808 13:15:54.329669 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.329679 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.329767 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.329848 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.331244 (+ 1396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37434: BlockingRecv error: recv got EOF from 127.0.0.1:37434 (error 108)
Metrics: {"server-negotiator.queue_time_us":336}
W20260808 13:15:54.343091 23841 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.324636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.31.59.188:33279)
0808 13:15:54.324847 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.324853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.329977 (+ 5124us) server_negotiation.cc:407] Connection header received
0808 13:15:54.330103 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.330108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.330188 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.330287 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.342837 (+ 12550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44878: BlockingRecv error: recv got EOF from 127.0.0.1:44878 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 13:15:54.385035 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.371875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.31.59.189:35563)
0808 13:15:54.372073 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.372079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.382708 (+ 10629us) server_negotiation.cc:407] Connection header received
0808 13:15:54.383284 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.383292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.383442 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.383528 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.384721 (+ 1193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55284: BlockingRecv error: recv got EOF from 127.0.0.1:55284 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 13:15:54.435164 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.421694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.31.59.189:35563)
0808 13:15:54.421883 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.421890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.433076 (+ 11186us) server_negotiation.cc:407] Connection header received
0808 13:15:54.433511 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.433518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.433638 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.433730 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.434916 (+ 1186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55294: BlockingRecv error: recv got EOF from 127.0.0.1:55294 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 13:15:54.486331 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.473002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.31.59.189:35563)
0808 13:15:54.473202 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.473208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.483758 (+ 10550us) server_negotiation.cc:407] Connection header received
0808 13:15:54.484341 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.484347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.484476 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.484559 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.486068 (+ 1509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55308: BlockingRecv error: recv got EOF from 127.0.0.1:55308 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 13:15:54.535804 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.526192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.31.59.189:35563)
0808 13:15:54.526451 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.526461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.534101 (+ 7640us) server_negotiation.cc:407] Connection header received
0808 13:15:54.534309 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.534315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.534414 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.534515 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.535536 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55320: BlockingRecv error: recv got EOF from 127.0.0.1:55320 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260808 13:15:54.583958 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.575550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.31.59.189:35563)
0808 13:15:54.575767 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.575773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.582200 (+ 6427us) server_negotiation.cc:407] Connection header received
0808 13:15:54.582453 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.582459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.582556 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.582671 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.583708 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55324: BlockingRecv error: recv got EOF from 127.0.0.1:55324 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 13:15:54.607021 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.604917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.31.59.189:35563)
0808 13:15:54.605101 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.605107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.605530 (+ 423us) server_negotiation.cc:407] Connection header received
0808 13:15:54.605733 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.605738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.605813 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.605912 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.606741 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55330: BlockingRecv error: recv got EOF from 127.0.0.1:55330 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 13:15:54.645164 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.643136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.31.59.189:35563)
0808 13:15:54.643346 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.643352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.643737 (+ 385us) server_negotiation.cc:407] Connection header received
0808 13:15:54.643930 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.643936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.644028 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.644176 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.644880 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55338: BlockingRecv error: recv got EOF from 127.0.0.1:55338 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 13:15:54.714362 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.712531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.31.59.189:35563)
0808 13:15:54.712700 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.712705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.713094 (+ 389us) server_negotiation.cc:407] Connection header received
0808 13:15:54.713261 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.713266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.713341 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.713483 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.714192 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55344: BlockingRecv error: recv got EOF from 127.0.0.1:55344 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 13:15:54.848272 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:54.846413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.31.59.189:35563)
0808 13:15:54.846653 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:54.846659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:54.846911 (+ 252us) server_negotiation.cc:407] Connection header received
0808 13:15:54.847108 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:54.847112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:54.847186 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:54.847279 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:54.848029 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55352: BlockingRecv error: recv got EOF from 127.0.0.1:55352 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:15:54.965369 23875 catalog_manager.cc:1259] Loaded cluster ID: 8d5b711b72c5449897583b8cd607591f
I20260808 13:15:54.965513 23875 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: loading cluster ID for follower catalog manager: success
I20260808 13:15:54.968020 23875 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: acquiring CA information for follower catalog manager: success
I20260808 13:15:54.969826 23875 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3d5332710fb4bb2931f2346b2684cf5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 13:15:55.113130 23761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:55.111245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.31.59.189:35563)
0808 13:15:55.111474 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.111481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.111769 (+ 288us) server_negotiation.cc:407] Connection header received
0808 13:15:55.111941 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.111945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.112019 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.112106 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:55.112923 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55360: BlockingRecv error: recv got EOF from 127.0.0.1:55360 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:15:55.137598 23909 catalog_manager.cc:1259] Loaded cluster ID: 8d5b711b72c5449897583b8cd607591f
I20260808 13:15:55.137703 23909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: loading cluster ID for follower catalog manager: success
I20260808 13:15:55.140185 23909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: acquiring CA information for follower catalog manager: success
I20260808 13:15:55.141994 23909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ecf325b2cddc4a44bf6aa1f92ae53468: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:15:55.634670 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:55.629072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.31.59.189:35563)
0808 13:15:55.629568 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.629573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.629611 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:15:55.629810 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.629814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.629981 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.630101 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:55.630499 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.631217 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.632503 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.632944 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.633040 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:55.633544 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:55.634107 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:55.634218 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:55.634277 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 13:15:55.634365 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":221,"threads_started":1}
I20260808 13:15:55.646342 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:55.640649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.31.59.188:33279)
0808 13:15:55.641140 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.641145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.641357 (+ 212us) server_negotiation.cc:407] Connection header received
0808 13:15:55.641571 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.641575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.641725 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.641867 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:55.642185 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.642885 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.644189 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.644632 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.644724 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:55.645178 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:55.645760 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:55.645873 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:55.645936 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 13:15:55.646002 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":227,"threads_started":1}
I20260808 13:15:55.672204 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:55.653199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.31.59.188:33279)
0808 13:15:55.658343 (+ 5144us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.658350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.666532 (+ 8182us) server_negotiation.cc:407] Connection header received
0808 13:15:55.666670 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.666675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.666760 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.666843 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:55.667330 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.668313 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.670432 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.671025 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.671137 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:55.671485 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:55.671632 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:55.671709 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:55.671843 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 13:15:55.671924 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4994}
I20260808 13:15:55.673019 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:55.652678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.31.59.190:39069)
0808 13:15:55.665918 (+ 13240us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.665924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.665945 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:15:55.666095 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.666101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.666336 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.666536 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:55.667531 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.668346 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.670139 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.670652 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.671561 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:55.672010 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:55.672437 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:55.672549 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:55.672655 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 13:15:55.672749 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13110,"thread_start_us":13200,"threads_started":1}
I20260808 13:15:55.673871 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:55.653208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.31.59.189:35563)
0808 13:15:55.653390 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:55.653396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:55.666205 (+ 12809us) server_negotiation.cc:407] Connection header received
0808 13:15:55.666415 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:55.666419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:55.666490 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:55.666578 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:15:55.667817 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.668819 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.670777 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:55.671323 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:55.671423 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:55.672468 (+ 1045us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:15:55.672585 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:15:55.672670 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:55.673522 (+ 852us) server_negotiation.cc:299] Negotiation successful
0808 13:15:55.673604 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:15:55.678292 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23589
I20260808 13:15:55.679234 23661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:55.791137 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23589
I20260808 13:15:55.814198 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23675
I20260808 13:15:55.815310 23750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:55.925215 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23675
I20260808 13:15:55.949860 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 23764
I20260808 13:15:55.950963 23832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:15:56.061270 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 23764
2026-08-08T13:15:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5190 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:15:56 dist-test-slave-49h3 krb5kdc[24014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:15:56 dist-test-slave-49h3 krb5kdc[24014](info): set up 2 sockets
Aug 08 13:15:56 dist-test-slave-49h3 krb5kdc[24014](info): commencing operation
krb5kdc: starting...
W20260808 13:15:57.185153 31982 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/test-user.keytab.
Aug 08 13:15:57 dist-test-slave-49h3 krb5kdc[24014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:15:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
Entry for principal kudu/127.31.59.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab.
I20260808 13:15:57.521682 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38599
--webserver_interface=127.31.59.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:15:57.860564 24035 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:57.860877 24035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:57.860982 24035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:57.861105 24035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:57.868378 24035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:57.868477 24035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:57.868579 24035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:57.868649 24035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:57.868736 24035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:57.878872 24035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.190.keytab
--principal=kudu/127.31.59.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.190:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:57.880237 24035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:57.881659 24035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:57.892899 24040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:57.892931 24041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:57.892940 24044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:57.893813 24035 server_base.cc:1034] running on GCE node
I20260808 13:15:57.894585 24035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:57.895949 24035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:57.897172 24035 hybrid_clock.cc:648] HybridClock initialized: now 1786194957897105 us; error 33 us; skew 500 ppm
I20260808 13:15:57.897518 24035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:57 dist-test-slave-49h3 krb5kdc[24014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:57.900563 24035 init.cc:377] Logged in from keytab as kudu/127.31.59.190@KRBTEST.COM (short username kudu)
I20260808 13:15:57.902068 24035 webserver.cc:460] Webserver started at http://127.31.59.190:40893/ using document root <none> and password file <none>
I20260808 13:15:57.902738 24035 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:57.902851 24035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:57.903185 24035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:57.906275 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data/instance:
uuid: "4fd2dc3c8dc843b8950e4f814a6054be"
format_stamp: "Formatted at 2026-08-08 13:15:57 on dist-test-slave-49h3"
server_key: "77878d3852cf59875d258a9dba26d487"
server_key_iv: "1bdc03ad2e9273304018d2c8cdd0a19a"
server_key_version: "encryptionkey@0"
I20260808 13:15:57.907053 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance:
uuid: "4fd2dc3c8dc843b8950e4f814a6054be"
format_stamp: "Formatted at 2026-08-08 13:15:57 on dist-test-slave-49h3"
server_key: "77878d3852cf59875d258a9dba26d487"
server_key_iv: "1bdc03ad2e9273304018d2c8cdd0a19a"
server_key_version: "encryptionkey@0"
I20260808 13:15:57.912058 24035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:15:57.915419 24051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:57.916651 24035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:57.916805 24035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal
uuid: "4fd2dc3c8dc843b8950e4f814a6054be"
format_stamp: "Formatted at 2026-08-08 13:15:57 on dist-test-slave-49h3"
server_key: "77878d3852cf59875d258a9dba26d487"
server_key_iv: "1bdc03ad2e9273304018d2c8cdd0a19a"
server_key_version: "encryptionkey@0"
I20260808 13:15:57.916994 24035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:57.942173 24035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:57.946699 24035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:57.946978 24035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:57.964751 24035 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.190:38599
I20260808 13:15:57.964774 24111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.190:38599 every 8 connection(s)
I20260808 13:15:57.966171 24035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/data/info.pb
I20260808 13:15:57.969907 24112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:57.973852 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 24035
I20260808 13:15:57.974040 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-0/wal/instance
I20260808 13:15:57.974469 31982 external_mini_cluster.cc:1428] Setting key 5dada71278e573ad770fa0b7900cfead
I20260808 13:15:57.976876 24112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:57 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:58.000140 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:57.980348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.31.59.190:38599)
0808 13:15:57.982073 (+ 1725us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:57.982081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:57.982137 (+ 56us) server_negotiation.cc:407] Connection header received
0808 13:15:57.983743 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:57.983751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:57.984053 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:57.984275 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:57.985395 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:57.987059 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:57.988422 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:57.989179 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:57.989355 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:57.992836 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:57.992844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:57.992856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:57.992891 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:57.995624 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:57.996434 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:57.996442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:57.996446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:57.996600 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:57.996962 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:57.996967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:57.996971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:57.997498 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:57.997689 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:57.998248 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 13:15:57.998609 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":1519,"spinlock_wait_cycles":27904,"thread_start_us":1312,"threads_started":1}
I20260808 13:15:58.000159 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:57.980352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38599 (local address 127.0.0.1:43642)
0808 13:15:57.981327 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:57.981393 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:57.983379 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:57.984271 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:57.984276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:57.984853 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:57.985190 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:57.985207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:57.987200 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:57.987204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:57.988286 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:57.988299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:57.989385 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:57.989390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:57.989557 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:57.990204 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:57.990225 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:57.992617 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:57.995770 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:57.995781 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:57.995793 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:57.996321 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:57.996704 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:57.996709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:57.996713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:57.996865 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:57.997641 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:57.997650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:57.997880 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 13:15:57.998105 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 13:15:57.998738 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":163,"thread_start_us":415,"threads_started":1}
I20260808 13:15:58.003331 24112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:58.004470 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.004022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40254)
0808 13:15:58.004193 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.004344 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 13:15:58.004738 24052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
I20260808 13:15:58.006330 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.005974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40264)
0808 13:15:58.006129 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.006195 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:15:58.006522 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:39151: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.59.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
Entry for principal kudu/127.31.59.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab.
I20260808 13:15:58.046618 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:39151
--webserver_interface=127.31.59.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:15:58.055888 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } attempt: 1
I20260808 13:15:58.057185 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.056662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40268)
0808 13:15:58.056878 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.057017 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:15:58.058238 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.057862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40272)
0808 13:15:58.057998 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.058078 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 13:15:58.058432 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:39151: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
I20260808 13:15:58.126660 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } attempt: 2
I20260808 13:15:58.127741 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.127267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40274)
0808 13:15:58.127511 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.127601 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:15:58.128698 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.128343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40286)
0808 13:15:58.128480 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.128565 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 13:15:58.128883 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:39151: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
I20260808 13:15:58.220139 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } attempt: 3
I20260808 13:15:58.221328 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.220911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40298)
0808 13:15:58.221091 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.221183 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 13:15:58.222324 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.221956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40308)
0808 13:15:58.222103 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.222173 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 13:15:58.222504 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:39151: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
I20260808 13:15:58.365744 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } attempt: 4
I20260808 13:15:58.366906 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.366373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40314)
0808 13:15:58.366677 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.366768 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 13:15:58.367944 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.367600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40326)
0808 13:15:58.367741 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.367823 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:15:58.368144 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.189:39151: Network error: Client connection negotiation failed: client connection to 127.31.59.189:39151: connect: Connection refused (error 111)
W20260808 13:15:58.391685 24119 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:58.392035 24119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:58.392138 24119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:58.392261 24119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:58.399632 24119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:58.399739 24119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:58.399819 24119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:58.399890 24119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:58.399974 24119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:58.410650 24119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.189.keytab
--principal=kudu/127.31.59.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.189:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:58.412019 24119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:58.413394 24119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:58.424777 24128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:58.425226 24125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:58.425196 24126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:58.426074 24119 server_base.cc:1034] running on GCE node
I20260808 13:15:58.426834 24119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:58.428272 24119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:58.429508 24119 hybrid_clock.cc:648] HybridClock initialized: now 1786194958429421 us; error 36 us; skew 500 ppm
I20260808 13:15:58.429827 24119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:58 dist-test-slave-49h3 krb5kdc[24014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:58.432847 24119 init.cc:377] Logged in from keytab as kudu/127.31.59.189@KRBTEST.COM (short username kudu)
I20260808 13:15:58.434247 24119 webserver.cc:460] Webserver started at http://127.31.59.189:43943/ using document root <none> and password file <none>
I20260808 13:15:58.434895 24119 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:58.434983 24119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:58.435248 24119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:58.438356 24119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data/instance:
uuid: "cd2fd53beb43474fa6b0bac7b1476f0f"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "c4f4a45363e0facf44a118dbcdab8ce0"
server_key_iv: "045b84bc52b42f218278ac7927d83a70"
server_key_version: "encryptionkey@0"
I20260808 13:15:58.439137 24119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal/instance:
uuid: "cd2fd53beb43474fa6b0bac7b1476f0f"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "c4f4a45363e0facf44a118dbcdab8ce0"
server_key_iv: "045b84bc52b42f218278ac7927d83a70"
server_key_version: "encryptionkey@0"
I20260808 13:15:58.444108 24119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:15:58.447476 24136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:58.448853 24119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:15:58.448989 24119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal
uuid: "cd2fd53beb43474fa6b0bac7b1476f0f"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "c4f4a45363e0facf44a118dbcdab8ce0"
server_key_iv: "045b84bc52b42f218278ac7927d83a70"
server_key_version: "encryptionkey@0"
I20260808 13:15:58.449206 24119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:58.492501 24119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:58.496413 24119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:58.496646 24119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:58.514658 24119 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.189:39151
I20260808 13:15:58.514680 24196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.189:39151 every 8 connection(s)
I20260808 13:15:58.516021 24119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/data/info.pb
I20260808 13:15:58.519603 24197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:58.519734 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 24119
I20260808 13:15:58.519924 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-1/wal/instance
I20260808 13:15:58.520413 31982 external_mini_cluster.cc:1428] Setting key eede8e7949cad0e56e8b32f1e781a6ca
I20260808 13:15:58.526422 24197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:58 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:58.547683 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.529041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.31.59.190:38599)
0808 13:15:58.529615 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:58.529621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:58.530539 (+ 918us) server_negotiation.cc:407] Connection header received
0808 13:15:58.532867 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:58.532871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:58.532940 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:58.533034 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:58.534997 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.536065 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.537607 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.538193 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.538358 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.541945 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:58.541950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:58.541954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:58.542000 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:58.544599 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.545467 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.545473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.545476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.545595 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.546006 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.546009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.546010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.546315 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:58.546423 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:58.547230 (+ 807us) server_negotiation.cc:299] Negotiation successful
0808 13:15:58.547409 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":282,"threads_started":1}
I20260808 13:15:58.549070 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.529580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38599 (local address 127.0.0.1:43650)
0808 13:15:58.530359 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.530414 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:58.532582 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:58.533297 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:58.533307 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:58.534061 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:58.534790 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.534814 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.536232 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.536237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.537421 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.537434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.538425 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.538430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.538644 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.539310 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:58.539337 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:58.541705 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:58.544751 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.544760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.544777 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.545347 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.545693 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.545698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.545701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.545916 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.546443 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:58.546450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:58.546877 (+ 427us) client_negotiation.cc:769] Sending connection context
0808 13:15:58.547178 (+ 301us) client_negotiation.cc:240] Negotiation successful
0808 13:15:58.547687 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":282,"threads_started":1}
I20260808 13:15:58.550938 24197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.59.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.188@KRBTEST.COM" created.
Aug 08 13:15:58 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:58.565738 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.551733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40330)
0808 13:15:58.551943 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.551962 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:58.552078 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:58.553078 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:58.553083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:58.553526 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:58.553716 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.553727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.555104 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.555109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.556017 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.556029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.557188 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.557194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.557373 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.557943 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:58.557964 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:58.560274 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:58.563308 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.563312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.563316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.563737 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.564083 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.564086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.564089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.564232 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.564805 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:58.564809 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:58.564943 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 13:15:58.565080 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 13:15:58.565356 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 13:15:58.565814 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.551999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.31.59.189:39151)
0808 13:15:58.552651 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:58.552657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:58.552682 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:15:58.552723 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:58.552728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:58.552933 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:58.553069 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:58.553876 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.554980 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.556144 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.557038 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.557153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.560480 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:58.560492 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:58.560505 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:58.560537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:58.563184 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.563858 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.563865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.563867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.563979 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.564342 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.564345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.564347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.564684 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:58.564848 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:58.565196 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 13:15:58.565484 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":515,"thread_start_us":369,"threads_started":1}
I20260808 13:15:58.568421 24197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:58.569522 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.569010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44856)
0808 13:15:58.569171 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.569316 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:15:58.569769 24140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.571257 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.570884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44864)
0808 13:15:58.571045 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.571133 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:15:58.571465 24197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
Entry for principal kudu/127.31.59.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab.
I20260808 13:15:58.594432 31982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskePcvnh/build/asan/bin/kudu
/tmp/dist-test-taskePcvnh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:34333
--webserver_interface=127.31.59.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.148:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:15:58.620812 24197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 1
I20260808 13:15:58.622005 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.621558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44870)
0808 13:15:58.621746 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.621856 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:15:58.623024 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.622695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44880)
0808 13:15:58.622842 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.622920 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:15:58.623226 24197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.667368 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } attempt: 5
Aug 08 13:15:58 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:58.680547 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.668172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.31.59.189:39151)
0808 13:15:58.668396 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:58.668402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:58.668425 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:15:58.668609 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:58.668614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:58.668679 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:58.668764 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:58.669597 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.670299 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.671480 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.671920 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.672029 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.675557 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:58.675562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:58.675565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:58.675600 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:58.678107 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.678890 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.678893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.678895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.679006 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:58.679382 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:58.679385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:58.679387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:58.679646 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:58.679750 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:58.680159 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 13:15:58.680332 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 13:15:58.680763 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.668121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40346)
0808 13:15:58.668335 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.668355 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:58.668485 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:58.668834 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:58.668839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:58.669221 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:58.669417 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.669426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.670447 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.670452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.671314 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:58.671326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:58.672061 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:58.672067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:58.672262 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:58.672839 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:58.672858 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:58.675364 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:58.678240 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.678245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.678249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.678788 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.679101 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:58.679106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:58.679109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:58.679285 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:58.679755 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:58.679759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:58.679932 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 13:15:58.680087 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 13:15:58.680419 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 13:15:58.682039 24112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:58.683157 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.682692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44886)
0808 13:15:58.682877 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.682978 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 13:15:58.684051 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.683699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44900)
0808 13:15:58.683828 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.683926 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 13:15:58.684248 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.691411 24197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 2
I20260808 13:15:58.692340 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.691953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44912)
0808 13:15:58.692116 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.692195 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 13:15:58.693306 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.692983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44920)
0808 13:15:58.693110 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.693182 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:15:58.693527 24197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.735525 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 1
I20260808 13:15:58.736855 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.736278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44932)
0808 13:15:58.736505 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.736628 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 13:15:58.738001 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.737595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44934)
0808 13:15:58.737760 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.737859 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 13:15:58.738243 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.784730 24197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 3
I20260808 13:15:58.786000 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.785525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44936)
0808 13:15:58.785736 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.785838 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 13:15:58.787171 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.786741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44946)
0808 13:15:58.786979 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.787055 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260808 13:15:58.787379 24197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.806483 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 2
I20260808 13:15:58.807794 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.807202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44962)
0808 13:15:58.807453 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.807569 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 13:15:58.808807 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.808410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44964)
0808 13:15:58.808574 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.808671 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 13:15:58.809038 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.915274 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 3
I20260808 13:15:58.916489 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.915920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44976)
0808 13:15:58.916145 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.916264 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 13:15:58.917606 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.917126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:44990)
0808 13:15:58.917336 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.917437 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 13:15:58.917867 24112 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
I20260808 13:15:58.930565 24197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 4
I20260808 13:15:58.931619 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.931163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:45006)
0808 13:15:58.931383 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.931466 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 13:15:58.932581 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:58.932245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:45014)
0808 13:15:58.932397 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:58.932464 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 13:15:58.932829 24197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.188:34333: Network error: Client connection negotiation failed: client connection to 127.31.59.188:34333: connect: Connection refused (error 111)
W20260808 13:15:58.945292 24205 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:15:58.945704 24205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:15:58.945822 24205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:15:58.945935 24205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:15:58.953601 24205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:15:58.953716 24205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:15:58.953810 24205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:15:58.953874 24205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:15:58.953961 24205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:15:58.964207 24205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.148:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.190:38599,127.31.59.189:39151,127.31.59.188:34333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/krb5kdc/kudu_127.31.59.188.keytab
--principal=kudu/127.31.59.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.188:34333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:15:58.965607 24205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:15:58.966977 24205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:15:58.978725 24210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:58.979082 24211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:15:58.979537 24214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:15:58.979975 24205 server_base.cc:1034] running on GCE node
I20260808 13:15:58.980813 24205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:15:58.982298 24205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:15:58.983552 24205 hybrid_clock.cc:648] HybridClock initialized: now 1786194958983464 us; error 39 us; skew 500 ppm
I20260808 13:15:58.983894 24205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:15:58 dist-test-slave-49h3 krb5kdc[24014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:15:58.987152 24205 init.cc:377] Logged in from keytab as kudu/127.31.59.188@KRBTEST.COM (short username kudu)
I20260808 13:15:58.988623 24205 webserver.cc:460] Webserver started at http://127.31.59.188:45639/ using document root <none> and password file <none>
I20260808 13:15:58.989313 24205 fs_manager.cc:359] Metadata directory not provided
I20260808 13:15:58.989404 24205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:15:58.989727 24205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:15:58.992906 24205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data/instance:
uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "3107e9d47ce632814b66a377eafc5943"
server_key_iv: "63fcfaeab793aacf18ca0a548803a5db"
server_key_version: "encryptionkey@0"
I20260808 13:15:58.993754 24205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal/instance:
uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "3107e9d47ce632814b66a377eafc5943"
server_key_iv: "63fcfaeab793aacf18ca0a548803a5db"
server_key_version: "encryptionkey@0"
I20260808 13:15:58.998927 24205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:15:59.002347 24222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:15:59.003832 24205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:15:59.004012 24205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data,/tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal
uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5"
format_stamp: "Formatted at 2026-08-08 13:15:58 on dist-test-slave-49h3"
server_key: "3107e9d47ce632814b66a377eafc5943"
server_key_iv: "63fcfaeab793aacf18ca0a548803a5db"
server_key_version: "encryptionkey@0"
I20260808 13:15:59.004237 24205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:15:59.042094 24205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:15:59.046202 24205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:15:59.046471 24205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:15:59.064913 24205 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.188:34333
I20260808 13:15:59.064939 24278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.188:34333 every 8 connection(s)
I20260808 13:15:59.066325 24205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/data/info.pb
I20260808 13:15:59.066715 31982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskePcvnh/build/asan/bin/kudu as pid 24205
I20260808 13:15:59.066881 31982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskePcvnh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194589892156-31982-0/minicluster-data/master-2/wal/instance
I20260808 13:15:59.067410 31982 external_mini_cluster.cc:1428] Setting key 1b2dc3fe56cc18ab614c895dc0d67369
I20260808 13:15:59.070565 24279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:15:59.077371 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
I20260808 13:15:59.088918 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.069218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.31.59.188:34333)
0808 13:15:59.069966 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.069977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.070017 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:15:59.071696 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.071718 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.072092 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.072440 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.073901 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.075241 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.076506 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.077175 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.077344 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.081650 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.081657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.081667 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.081705 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.084848 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.085482 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.085491 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.085499 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.085716 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.086130 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.086135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.086138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.086715 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.086963 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.087362 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.087743 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"spinlock_wait_cycles":12032,"thread_start_us":319,"threads_started":1}
I20260808 13:15:59.094841 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.079772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.190:38599 (local address 127.0.0.1:43654)
0808 13:15:59.080693 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:59.080733 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:59.080888 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:59.081252 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:59.081259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:59.081975 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:59.082292 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.082305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.083666 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.083670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.084770 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.084783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.085576 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.085580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.085711 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.086320 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:59.086337 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:59.088757 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:59.092393 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.092406 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.092411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.092837 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.093251 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.093256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.093259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.093419 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.093902 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:59.093909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:59.094129 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 13:15:59.094309 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 13:15:59.094549 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":545,"threads_started":1}
I20260808 13:15:59.095098 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.079825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.31.59.190:38599)
0808 13:15:59.080347 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.080353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.080824 (+ 471us) server_negotiation.cc:407] Connection header received
0808 13:15:59.081005 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.081009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.081085 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.081178 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.082456 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.083506 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.084925 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.085363 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.085516 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.088955 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.088962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.088966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.089009 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.092238 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.092963 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.092968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.092972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.093140 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.093533 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.093536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.093538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.093784 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.093907 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.094431 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.094664 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":239,"threads_started":1}
I20260808 13:15:59.095216 24112 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 4
I20260808 13:15:59.096709 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.190@KRBTEST.COM
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.190@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:59.110308 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.092086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.31.59.190:38599)
0808 13:15:59.092647 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.092651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.092669 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:15:59.092706 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.092709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.092768 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.092847 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.094346 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.095591 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.096915 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.097648 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.097758 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.102425 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.102432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.102437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.102470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.105609 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.106416 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.106422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.106425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.106557 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.106904 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.106908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.106911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.107270 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.107535 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.109643 (+ 2108us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.109868 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":267,"threads_started":1}
I20260808 13:15:59.111945 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.097384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.189:39151 (local address 127.0.0.1:40360)
0808 13:15:59.097805 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:59.097821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:59.097926 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:59.098217 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:59.098222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:59.098737 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:59.098913 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.098924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.100039 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.100044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.101074 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.101090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.102264 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.102270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.102425 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.102902 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:59.102917 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:59.105502 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:59.108650 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.108655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.108658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.109311 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.109926 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.109930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.109932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.110178 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.110955 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:59.110960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:59.111126 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 13:15:59.111307 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 13:15:59.111595 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260808 13:15:59.112007 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.097514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.31.59.189:39151)
0808 13:15:59.097684 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.097688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.097907 (+ 219us) server_negotiation.cc:407] Connection header received
0808 13:15:59.098024 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.098028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.098090 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.098162 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.099095 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.099892 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.101256 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.101907 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.102005 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.105731 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.105736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.105739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.105772 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.108506 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.109416 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.109422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.109426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.109622 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.110313 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.110318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.110322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.110731 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.110862 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.111430 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.111653 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:15:59.112033 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.095881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:45034)
0808 13:15:59.096027 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:59.096044 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:59.096131 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:59.096900 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:59.096905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:59.097397 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:59.097623 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.097635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.099450 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.099455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.100804 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.100817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.101566 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.101571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.101735 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.102231 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:59.102263 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:59.105556 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:59.109126 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.109132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.109136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.109741 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.110210 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.110215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.110218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.110422 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.111106 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:59.111111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:59.111301 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 13:15:59.111423 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:15:59.111680 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260808 13:15:59.112875 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.096061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.31.59.188:34333)
0808 13:15:59.096610 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.096615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.096632 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:15:59.096675 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.096679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.096752 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.096844 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.098303 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.099308 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.100936 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.101414 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.101554 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.105743 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.105749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.105754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.105789 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.108957 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.109858 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.109862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.109865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.110084 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.110563 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.110568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.110571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.110911 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.111038 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.112444 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.112610 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444}
I20260808 13:15:59.113271 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:15:59.119552 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: Bootstrap starting.
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.188@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.189@KRBTEST.COM
I20260808 13:15:59.124013 24112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:59.125170 24112 log.cc:824] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:59.127586 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.114016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:45036)
0808 13:15:59.114175 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:59.114194 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:59.114290 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:59.114578 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:59.114582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:59.114954 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:59.115116 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.115128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.116395 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.116399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.117696 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.117712 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.118629 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.118635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.118863 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.119416 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:59.119439 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:59.121648 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:59.124691 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.124694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.124697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.125145 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.125569 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.125574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.125577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.125726 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.126507 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:59.126512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:59.126669 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 13:15:59.126826 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 13:15:59.127164 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 13:15:59.127692 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.114170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.31.59.188:34333)
0808 13:15:59.114338 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.114343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.114358 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:15:59.114398 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.114402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.114466 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.114547 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.115255 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.116254 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.117852 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.118475 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.118607 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.121835 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.121840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.121843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.121870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.124528 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.125275 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.125279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.125283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.125422 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.125828 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.125833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.125837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.126290 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.126456 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.126977 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.127327 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":41}
I20260808 13:15:59.128654 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: No bootstrap required, opened a new log
I20260808 13:15:59.130825 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.112269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.31.59.189:39151)
0808 13:15:59.112707 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.112713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.113193 (+ 480us) server_negotiation.cc:407] Connection header received
0808 13:15:59.113249 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.113253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.113328 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.113425 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.115073 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.116138 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.117977 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.118562 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.118652 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.123985 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.123992 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.123998 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.124048 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.127589 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.128683 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.128689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.128692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.128856 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.129258 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.129263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.129267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.129634 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.129882 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.130185 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.130407 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326}
I20260808 13:15:59.133598 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: Bootstrap starting.
I20260808 13:15:59.135049 31982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:15:59.137283 24084 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4fd2dc3c8dc843b8950e4f814a6054be"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43668
I20260808 13:15:59.137535 24279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:59.138885 24279 log.cc:824] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:59.139320 24112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.139719 24112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd2dc3c8dc843b8950e4f814a6054be, State: Initialized, Role: FOLLOWER
I20260808 13:15:59.140169 24112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.141088 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.141532 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.142272 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: No bootstrap required, opened a new log
I20260808 13:15:59.142454 24112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:59.142536 24084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 13:15:59.142745 24084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:59.145247 24084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.146476 24084 leader_election.cc:288] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [CANDIDATE]: Term 1 election: Requested vote from peers cd2fd53beb43474fa6b0bac7b1476f0f (127.31.59.189:39151), 0bd732e2bf66451ea5d6d4e6187fbbe5 (127.31.59.188:34333)
I20260808 13:15:59.153703 24279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.154170 24279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd732e2bf66451ea5d6d4e6187fbbe5, State: Initialized, Role: FOLLOWER
I20260808 13:15:59.154709 24279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.157119 24279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:59.157289 24252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5"
I20260808 13:15:59.157823 24252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:59.162900 24252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd2dc3c8dc843b8950e4f814a6054be in term 1.
I20260808 13:15:59.164673 24056 leader_election.cc:302] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd732e2bf66451ea5d6d4e6187fbbe5, 4fd2dc3c8dc843b8950e4f814a6054be; no voters:
I20260808 13:15:59.165150 24290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:15:59.171546 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.173956 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.184872 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.156009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.31.59.190:38599)
0808 13:15:59.156216 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.156222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.157151 (+ 929us) server_negotiation.cc:407] Connection header received
0808 13:15:59.157470 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.157476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.157573 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.157653 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.159943 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.161111 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.163103 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.163771 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.163873 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.176295 (+ 12422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.176303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.176308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.176358 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.180096 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.181415 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.181422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.181425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.181629 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.182516 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.182521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.182525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.182911 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.183190 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.184242 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.184499 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:15:59.187119 24290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [term 1 LEADER]: Becoming Leader. State: Replica: 4fd2dc3c8dc843b8950e4f814a6054be, State: Running, Role: LEADER
I20260808 13:15:59.187872 24290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
W20260808 13:15:59.199851 24058 tablet.cc:2369] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:15:59.201469 24309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fd2dc3c8dc843b8950e4f814a6054be. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.201990 24309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:59.211550 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.176345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45044 (local address 127.31.59.188:34333)
0808 13:15:59.176524 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.176540 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.186107 (+ 9567us) server_negotiation.cc:407] Connection header received
0808 13:15:59.191077 (+ 4970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.191083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.191203 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.191280 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.193481 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.194535 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.196603 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.197299 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.197393 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.202404 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.202411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.202415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.202459 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.206290 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.207594 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.207613 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.207617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.207775 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.208523 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.208527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.208530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.208993 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.209419 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.210528 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.210922 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 13:15:59.221746 24328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:15:59.221987 24328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:15:59.206171 24035 master_runner.cc:186] Error getting master registration for 127.31.59.189:39151: OK, instance_id { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" instance_seqno: 1786194958509646 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:15:59.225133 24035 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:15:59.230762 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.156933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.31.59.189:39151)
0808 13:15:59.157117 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.157123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.177719 (+ 20596us) server_negotiation.cc:407] Connection header received
0808 13:15:59.178099 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.178106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.178202 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.178282 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.184918 (+ 6636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.185979 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.191674 (+ 5695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.192324 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.192426 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.197387 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.197394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.197398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.197466 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.208698 (+ 11232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.226341 (+ 17643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.226348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.226352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.226715 (+ 363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.227513 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.227539 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.227543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.228310 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.228696 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.229693 (+ 997us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.230100 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:15:59.231999 24323 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 13:15:59.232012 24197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } attempt: 5
W20260808 13:15:59.232131 24323 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:15:59.232275 24324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:15:59.222223 24205 master_runner.cc:186] Error getting master registration for 127.31.59.189:39151: OK, instance_id { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" instance_seqno: 1786194958509646 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:15:59.235430 24205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:15:59.235880 24324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 13:15:59 dist-test-slave-49h3 krb5kdc[24014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.189@KRBTEST.COM for kudu/127.31.59.188@KRBTEST.COM
I20260808 13:15:59.249255 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.232577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.188:34333 (local address 127.0.0.1:45050)
0808 13:15:59.232769 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 13:15:59.232786 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:15:59.232885 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:15:59.233467 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:15:59.233472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:15:59.233934 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:15:59.234139 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.234152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.235789 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.235795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.237118 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:15:59.237135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.238414 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.238421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:15:59.238648 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.239354 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:15:59.239374 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:15:59.242237 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:15:59.246407 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.246412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.246416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.246943 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.247428 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:15:59.247433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:15:59.247437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:15:59.247608 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:15:59.248296 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:15:59.248301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:15:59.248463 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 13:15:59.248591 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 13:15:59.248873 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 13:15:59.250528 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:15:59.232941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.31.59.188:34333)
0808 13:15:59.233123 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.233129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.233148 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:15:59.233203 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.233208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.233285 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.233392 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.234392 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.235501 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.237348 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:15:59.238056 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:15:59.238185 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:15:59.242447 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:15:59.242453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:15:59.242458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:15:59.242500 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:15:59.246258 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.247073 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.247078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.247082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.247281 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:15:59.247742 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:15:59.247747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:15:59.247750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:15:59.248122 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:15:59.249649 (+ 1527us) server_negotiation.cc:1035] Waiting for connection context
0808 13:15:59.249902 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 13:15:59.250187 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:15:59.252506 24252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [term 1 FOLLOWER]: Refusing update from remote peer 4fd2dc3c8dc843b8950e4f814a6054be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:59.253611 24309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:59.257231 24197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: Bootstrap starting.
I20260808 13:15:59.261648 24197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: Neither blocks nor log segments found. Creating new log.
I20260808 13:15:59.262951 24197 log.cc:824] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:15:59.277809 24197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: No bootstrap required, opened a new log
I20260808 13:15:59.282684 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fd2dc3c8dc843b8950e4f814a6054be. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.283049 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.287917 24330 mvcc.cc:205] Tried to move back new op lower bound from 7316254553085538304 to 7316254552840712192. Current Snapshot: MvccSnapshot[applied={T|T < 7316254553085538304}]
I20260808 13:15:59.293898 24309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.294229 24309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:59.296011 24324 catalog_manager.cc:1347] Generated new cluster ID: 0472e337db1644f8a7f23fe02a75c9fd
I20260808 13:15:59.296132 24324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:15:59.301770 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.302058 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.310194 24197 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.310740 24197 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2fd53beb43474fa6b0bac7b1476f0f, State: Initialized, Role: FOLLOWER
I20260808 13:15:59.311362 24197 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } }
I20260808 13:15:59.314271 24197 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:15:59.317771 24168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f"
I20260808 13:15:59.318280 24168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:15:59.325021 24168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd2dc3c8dc843b8950e4f814a6054be in term 1.
I20260808 13:15:59.326818 24324 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:15:59.327925 24167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [term 1 FOLLOWER]: Refusing update from remote peer 4fd2dc3c8dc843b8950e4f814a6054be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:15:59.328725 24324 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:15:59.329713 24309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:15:59.362614 24324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be: Generated new TSK 0
I20260808 13:15:59.363797 24324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:15:59.434309 24337 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.434762 24337 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.435209 24341 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fd2dc3c8dc843b8950e4f814a6054be. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.435283 24350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.435632 24341 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.435707 24350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:15:59.437026 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd2dc3c8dc843b8950e4f814a6054be" member_type: VOTER last_known_addr { host: "127.31.59.190" port: 38599 } } peers { permanent_uuid: "cd2fd53beb43474fa6b0bac7b1476f0f" member_type: VOTER last_known_addr { host: "127.31.59.189" port: 39151 } } peers { permanent_uuid: "0bd732e2bf66451ea5d6d4e6187fbbe5" member_type: VOTER last_known_addr { host: "127.31.59.188" port: 34333 } } }
I20260808 13:15:59.437363 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd2dc3c8dc843b8950e4f814a6054be [sys.catalog]: This master's current role is: LEADER
I20260808 13:15:59.446271 24357 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 13:15:59.484603 31982 mini_kdc.cc:348] Time spent kdestroy: real 0.102s user 0.000s sys 0.031s
W20260808 13:15:59.492674 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.488544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.31.59.189:39151)
0808 13:15:59.488756 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.488762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.490211 (+ 1449us) server_negotiation.cc:407] Connection header received
0808 13:15:59.491075 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.491083 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.491175 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.491260 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.492432 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40392: BlockingRecv error: recv got EOF from 127.0.0.1:40392 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 13:15:59.492884 24287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.489117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.31.59.190:38599)
0808 13:15:59.489317 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.489323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.490562 (+ 1239us) server_negotiation.cc:407] Connection header received
0808 13:15:59.491315 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.491321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.491407 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.491477 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.492602 (+ 1125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43684: BlockingRecv error: recv got EOF from 127.0.0.1:43684 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 13:15:59.495139 24284 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.490964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.31.59.188:34333)
0808 13:15:59.491187 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.491193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.493669 (+ 2476us) server_negotiation.cc:407] Connection header received
0808 13:15:59.493804 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.493810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.493894 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.493984 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.494888 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45064: BlockingRecv error: recv got EOF from 127.0.0.1:45064 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:15:59.502027 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.499884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.31.59.189:39151)
0808 13:15:59.500033 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.500038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.500563 (+ 525us) server_negotiation.cc:407] Connection header received
0808 13:15:59.500758 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.500763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.500879 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.500989 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.501757 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40408: BlockingRecv error: recv got EOF from 127.0.0.1:40408 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 13:15:59.515780 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.511957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.31.59.189:39151)
0808 13:15:59.512137 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.512143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.514015 (+ 1872us) server_negotiation.cc:407] Connection header received
0808 13:15:59.514348 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.514355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.514455 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.514536 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.515563 (+ 1027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40412: BlockingRecv error: recv got EOF from 127.0.0.1:40412 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 13:15:59.554483 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.552067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40426 (local address 127.31.59.189:39151)
0808 13:15:59.552276 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.552282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.552869 (+ 587us) server_negotiation.cc:407] Connection header received
0808 13:15:59.553121 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.553127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.553232 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.553352 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.554244 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40426: BlockingRecv error: recv got EOF from 127.0.0.1:40426 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 13:15:59.565904 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.563489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.31.59.189:39151)
0808 13:15:59.563674 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.563679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.564224 (+ 545us) server_negotiation.cc:407] Connection header received
0808 13:15:59.564459 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.564465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.564562 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.564666 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.565659 (+ 993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40436: BlockingRecv error: recv got EOF from 127.0.0.1:40436 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:15:59.586383 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.583946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.31.59.189:39151)
0808 13:15:59.584150 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.584156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.584735 (+ 579us) server_negotiation.cc:407] Connection header received
0808 13:15:59.584956 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.584962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.585063 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.585176 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.586138 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40452: BlockingRecv error: recv got EOF from 127.0.0.1:40452 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:15:59.608948 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.606435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.31.59.189:39151)
0808 13:15:59.606655 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.606661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.607265 (+ 604us) server_negotiation.cc:407] Connection header received
0808 13:15:59.607505 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.607513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.607633 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.607772 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.608663 (+ 891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40456: BlockingRecv error: recv got EOF from 127.0.0.1:40456 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 13:15:59.647483 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.644972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40460 (local address 127.31.59.189:39151)
0808 13:15:59.645175 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.645182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.645768 (+ 586us) server_negotiation.cc:407] Connection header received
0808 13:15:59.645996 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.646003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.646128 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.646245 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.647210 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40460: BlockingRecv error: recv got EOF from 127.0.0.1:40460 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:15:59.716964 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.715001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.31.59.189:39151)
0808 13:15:59.715283 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.715290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.715581 (+ 291us) server_negotiation.cc:407] Connection header received
0808 13:15:59.715791 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.715795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.715870 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.715959 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.716744 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40470: BlockingRecv error: recv got EOF from 127.0.0.1:40470 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 13:15:59.853334 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:15:59.851297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.31.59.189:39151)
0808 13:15:59.851566 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 13:15:59.851573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:15:59.851820 (+ 247us) server_negotiation.cc:407] Connection header received
0808 13:15:59.852048 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:15:59.852055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:15:59.852147 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:15:59.852245 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:15:59.853075 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40472: BlockingRecv error: recv got EOF from 127.0.0.1:40472 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260808 13:16:00.117615 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:16:00.115725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.31.59.189:39151)
0808 13:16:00.115965 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.115971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.116174 (+ 203us) server_negotiation.cc:407] Connection header received
0808 13:16:00.116414 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.116418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.116495 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.116583 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:16:00.117360 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40482: BlockingRecv error: recv got EOF from 127.0.0.1:40482 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 13:16:00.224069 24328 catalog_manager.cc:1259] Loaded cluster ID: 0472e337db1644f8a7f23fe02a75c9fd
I20260808 13:16:00.224179 24328 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: loading cluster ID for follower catalog manager: success
I20260808 13:16:00.226786 24328 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: acquiring CA information for follower catalog manager: success
I20260808 13:16:00.228569 24328 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bd732e2bf66451ea5d6d4e6187fbbe5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:16:00.450489 24357 catalog_manager.cc:1259] Loaded cluster ID: 0472e337db1644f8a7f23fe02a75c9fd
I20260808 13:16:00.450613 24357 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: loading cluster ID for follower catalog manager: success
I20260808 13:16:00.453320 24357 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: acquiring CA information for follower catalog manager: success
I20260808 13:16:00.455693 24357 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd2fd53beb43474fa6b0bac7b1476f0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:16:00.639474 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:16:00.633254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.31.59.189:39151)
0808 13:16:00.634081 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.634086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.634101 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:16:00.634168 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.634171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.634310 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.634414 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:16:00.634882 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.635756 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.637108 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.637643 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.637767 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:16:00.638271 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:16:00.638815 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:16:00.638959 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 13:16:00.639061 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 13:16:00.639152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":507,"threads_started":1}
I20260808 13:16:00.653471 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:16:00.646681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.31.59.188:34333)
0808 13:16:00.647212 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.647218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.647571 (+ 353us) server_negotiation.cc:407] Connection header received
0808 13:16:00.647797 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.647801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.647950 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.648104 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:16:00.648590 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.649300 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.651185 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.651591 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.651682 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:16:00.652277 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:16:00.652846 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:16:00.652958 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:16:00.653070 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 13:16:00.653143 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260808 13:16:00.681685 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:16:00.660184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.31.59.190:38599)
0808 13:16:00.672569 (+ 12385us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.672577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.673316 (+ 739us) server_negotiation.cc:407] Connection header received
0808 13:16:00.673738 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.673744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.673953 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.674121 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:16:00.674979 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.676106 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.678354 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.678979 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.679085 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:16:00.679967 (+ 882us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:16:00.680431 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:16:00.680511 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:16:00.681230 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 13:16:00.681335 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12220,"thread_start_us":12402,"threads_started":1}
I20260808 13:16:00.683650 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:16:00.660408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.31.59.189:39151)
0808 13:16:00.660583 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.660589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.676008 (+ 15419us) server_negotiation.cc:407] Connection header received
0808 13:16:00.676523 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.676531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.676630 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.676690 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:16:00.677609 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.678898 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.681247 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.681966 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.682064 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:16:00.682747 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:16:00.682898 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:16:00.682963 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:16:00.683195 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 13:16:00.683300 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 13:16:00.688907 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:16:00.673750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.31.59.188:34333)
0808 13:16:00.673917 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 13:16:00.673923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:16:00.681141 (+ 7218us) server_negotiation.cc:407] Connection header received
0808 13:16:00.682462 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:16:00.682468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:16:00.682576 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:16:00.682641 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:16:00.683170 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.684221 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.687129 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:16:00.687684 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:16:00.687811 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:16:00.688201 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:16:00.688314 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:16:00.688391 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 13:16:00.688526 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 13:16:00.688611 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 13:16:00.690820 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 24035
I20260808 13:16:00.692340 24106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:16:00.805908 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 24035
I20260808 13:16:00.829818 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 24119
I20260808 13:16:00.830961 24192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:16:00.941474 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 24119
I20260808 13:16:00.970067 31982 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskePcvnh/build/asan/bin/kudu pid 24205
I20260808 13:16:00.971274 24274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:16:01.082532 31982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskePcvnh/build/asan/bin/kudu with pid 24205
2026-08-08T13:16:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5017 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10208 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (371153 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST